Metamorphic testing

Metamorphic testing (MT) is a property-based software testing technique, which can be an effective approach for addressing the test oracle problem and test case generation problem. The test oracle problem is the difficulty of determining the expected outcomes of selected test cases or to determine whether the actual outputs agree with the expected outcomes. Metamorphic relations (MRs) are necessary properties of the intended functionality of the software, and must involve multiple executions of the software. Consider, for example, a program that implements sin x correct to 100 significant figures; a metamorphic relation for sine functions is "sin (π − x) = sin x". Thus, even though the expected value of sin x₁ for the source test case x₁ = 1.234 correct to the required accuracy is not known, a follow-up test case x₂ = π − 1.234 can be constructed. We can verify whether the actual outputs produced by the program under test from the source test case and the follow-up test case are consistent with the MR in question. Any inconsistency (after taking rounding errors into consideration) indicates a failure^[1]:31 of the program, caused by a fault^[1]:31 in the implementation.

MRs are not limited to programs with numerical inputs or equality relations. As an example, when testing a booking website, a web search for accommodation in Sydney, Australia, returns 1,671 results; are the results of this search correct and complete? This is a test oracle problem. Based on a metamorphic relation, we may filter the price range or star rating and apply the search again; it should return a subset of the previous results. A violation of this expectation would similarly reveal a failure of the system.

Metamorphic testing was invented by T.Y. Chen in the technical report ^[2] in 1998. Since then, more than 150 international researchers and practitioners have applied the technique to real-life applications. Some examples include web services,^[3] computer graphics,^[4] embedded systems,^[5] simulation and modeling,^[6] machine learning,^[7] decision support,^[8] bioinformatics,^[9] components,^[10] numerical analysis,^[11] and compilers.^[12] The first major survey of the field of MT was conducted in 2016.^[13] It was followed by another major survey in 2018,^[14] which highlights the challenges and opportunities and clarifies common misunderstandings.

Although MT was initially proposed as a software verification technique, it was later developed into a paradigm that covers verification, validation, and other types of software quality assessment.^[15] MT can be applied independently, and can also be combined with other static and dynamic software analysis techniques such as proving and debugging.^[16]

In August 2018, Google acquired GraphicsFuzz, a startup from Imperial College London, to apply metamorphic testing to graphics device drivers for Android smartphones.^[17][18]

See also

• Mutation testing
• Property testing
• QuickCheck

References

External links

• "Metamorphic testing: Introduction and applications". https://event.on24.com/wcc/r/1451736/8B5B5925E82FC9807CF83C84834A6F3D. 
• Segura, Sergio; Zhou, Zhi Quan (2018). ICSE 2018 technical briefing on metamorphic testing. doi:10.5281/zenodo.1256230. 
• Zhou, Zhi Quan (2019). Automated testing of real-life self-driving systems and beyond: Keynote speech by Z.Q. (George) Zhou at IEEE AITest, San Francisco East Bay, 2019. doi:10.5281/zenodo.2636979. 

0.00 (0 votes) Original source: https://en.wikipedia.org/wiki/Metamorphic testing. Read more