Is mutation an appropriate tool for testing experiments? [software testing]

By J. Andrews, Lionel Claude Briand, Yvan Labiche. Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005., 2005

998 citations recorded by the source corpus.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/0cb937952b654695e5afdbc86e690894f34605fa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

Citations in the corpus

Taxonomy of Real Faults in Deep Learning Systems (2019, background): “Whether mutants are a valid substitute for real faults has been an ongoing debate for traditional software [11, 14, 18].”
TimeAware test suite prioritization (2006, background): “Even though this threat can be controlled by conducting additional case studies with a greater number of both seeded and real-world faults, it is interesting to remark that preliminary empirical studies suggest that mutation faults do correspond to the faults found in real-world applications [4].”
Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria (2006, background): “To conclude, except for our previous study [1], there has been no empirical study that has directly assessed the use of mutants by comparing them with results obtained on real faults.”
Evaluating and Improving Fault Localization (2017, result): “However, Namin et al. [32] studied the same set of programs as previous studies [6], and cautioned of the substantial external threats to validity when using mutants for experiments.”
Metallaxis‐FL: mutation‐based fault localization (2015, methodology): “Based on this requirement, mutants are able to simulate well the faults’ behavior [2].”
Mutation-Driven Generation of Unit Tests and Oracles (2010, methodology): “TEST optimizes test suites to detect mutants; a basic assumption of mutation testing is that mutants are based on actual fault models and are representative of real faults, and experiments have confirmed that generated mutants are indeed similar to real faults for the purpose of evaluating testing t”
The DStar Method for Effective Software Fault Localization (2014, background): “This creation was done to enlarge our data sets, and because mutation-based faults have been shown to simulate realistic faults well and provide reliable, trustworthy results [4], [11], [25], [28].”
Are mutants a valid substitute for real faults in software testing? (2014, methodology): “[8] 12 1,000 gen Rc,Ri,Ro 1% no [1] 38 5,905 gen Rc,Ro,Nbc,Ds 10% no [27] 38 5,905 gen Rc,Ri,Ro,Nbc,Ds 10% no”
A Survey on Software Fault Localization (2016, background): “Although mutation has been shown to be an effective approach to simulate realistic faults [29], [103], [223], [268], some real-life bugs are very delicate and cannot be modeled by simple ﬁrst-order mutants.”
Chapter Six - Mutation Testing Advances: An Analysis and Survey (2019, methodology): “Name & Ref Year Application Description mutate [335] N/A C supports method-level mutant operators Jester [336] 2001 Java supports source-code-level (src-level) mutant generation Proteum [337] 2001 C supports an extensive set of method-level mutant operators and interface mutation (inter-method level”
Higher Order Mutation Testing (2009, background): “[14] hand seeded faults vs mutants Space, Printtokens, Printtokens2, Replace, Schedule, Schedule2, Tcas and Totinfo Do and Rothermel [79, 80] hand seeded faults vs mutants Ant, Xml-security, Jmeter, Jtopas, galileo and nanoxml Li et ail.”
Testing and Validating Machine Learning Classifiers by Metamorphic Testing (2011, positive): “Mutation testing has been shown to be suitable for evaluation of effectiveness, as experiments comparing mutants to real faults have suggested that mutants are a good proxy for comparisons of testing techniques (Andrews et al., 2005).”
Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact (2005, background): “[3] indicates that mutation faults can in fact be representative of real faults.”