Metamorphic Testing: A New Approach for Generating Next Test Cases

By Tsong Yueh Chen, Shing-Chi Cheung, Siu Ming Yiu. arXiv.org, 2020

671 citations recorded by the source corpus.

In software testing, a set of test cases is constructed according to some predefined selection criteria. The software is then examined against these test cases. Three interesting observations have been made on the current artifacts of software testing. Firstly, an error-revealing test case is considered useful while a successful test case which does not reveal software errors is usually not further investigated. Whether these successful test cases still contain useful information for revealing software errors has not been properly studied. Secondly, no matter how extensive the testing has been conducted in the development phase, errors may still exist in the software [5]. These errors, if left undetected, may eventually cause damage to the production system. The study of techniques for uncovering software errors in the production phase is seldom addressed in the literature. Thirdly, as indicated by Weyuker in [6], the availability of test oracles is pragmatically unattainable in most situations. However, the availability of test oracles is generally assumed in conventional software testing techniques. In this paper, we propose a novel test case selection technique that derives new test cases from the successful ones. The selection aims at revealing software errors that are possibly left undetected in successful test cases which may be generated using some existing strategies. As such, the proposed technique augments the effectiveness of existing test selection strategies. The technique also helps uncover software errors in the production phase and can be used in the absence of test oracles.

Subfields: Information Systems and Management, Software

View Paper: https://www.semanticscholar.org/paper/4578871d2b271e4b5473c9cb81d431d6bf58c607

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Software

Citations in the corpus

Metamorphic Testing (2018, methodology): “Unlike most other soware testing techniques, Metamorphic Testing (MT ) [15] can be used for both test case generation and test result verication — thus addressing both fundamental problems of testing.”
DeepRoad: GAN-Based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems (2018, methodology): “Metamorphic testing is a classical software testing method that identifies software bugs [15, 36, 44].”
TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing (2018, positive): “They perform natural image transformations such as blurring, shearing, and transformation and use the idea of metamorphic testing [8] to ﬁnd errors.”
Machine Learning Testing: Survey, Landscapes and Horizons (2019, background): “[114] to ameliorate the test oracle problem in traditional software”
DeepHunter: a coverage-guided fuzz testing framework for deep neural networks (2019, methodology): “This work adopts the metamorphic relation of test image during transformation as the oracle [10].”