An orchestrated survey of methodologies for automated software test case generation

By Saswat Anand, Edmund Kieran Burke, Tsong Yueh Chen, John A. Clark, Myra B. Cohen, Wolfgang Grieskamp, Mark Harman, Mary Jean Harrold, Phil McMinn. Journal of Systems and Software, 2013

714 citations recorded by the source corpus.

Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software testing for several decades, resulting in many different approaches and tools. This paper presents an orchestrated survey of the most prominent techniques for automatic generation of software test cases, reviewed in self-standing sections. The techniques presented include: (a) structural testing using symbolic execution, (b) model-based testing, (c) combinatorial testing, (d) random testing and its variant of adaptive random testing, and (e) search-based testing. Each section is contributed by world-renowned active researchers on the technique, and briefly covers the basic ideas underlying the method, the current state of the art, a discussion of the open research problems, and a perspective of the future development of the approach. As a whole, the paper aims at giving an introductory, up-to-date and (relatively) short overview of research in automatic test case generation, while ensuring a comprehensive and authoritative treatment.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/1fab8830baef8f353dce995c0269a32fd3105f64

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

Citations in the corpus

Using Frankencerts for Automated Adversarial Testing of Certificate Validation in SSL/TLS Implementations (2014, background): “Georgiev et al. focus primarily on the incorrect validation of hostnames in server certificates at a different level in the software stack—in applications, transport libraries, and Webservices middleware.”
A Survey on Metamorphic Testing (2016, methodology): “This problem is referred to as the oracle problem and it is recognised as one of the fundamental challenges of software testing [1], [2], [3], [4].”
DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars (2017, neutral): “[27], McMinn et al.”
Large Language Models for Software Engineering: Survey and Open Problems (2023, background): “Clearly,automated testing techniques [3]–[5] will have a central role to play in ensuring correctness, just as they already do for human-engineered artefacts.”
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions (2024, neutral): “Furthermore, the function calls used to pass test cases by LLMs are more nondeterministic, making traditional test generation [7] methods cover all possible scenarios.”
Leveraging Automated Unit Tests for Unsupervised Code Translation (2021, background): “To reduce such cost, much research over the last three decades has focused on automating the process of test generation (Anand et al., 2013).”
The Art, Science, and Engineering of Fuzzing: A Survey (2018, background): “A broader treatment of dynamic symbolic execution can be found in other sources [20], [192].”
Agentic Software Issue Resolution with Large Language Models: A Survey (2025, unknown): “In recent years, LLMs have achieved significant success in multiple areas of software engineering [10, 20, 165] due to their rapid advancements in code understanding [138], reasoning [57, 162], and generation [73], paving the way for automated issue resolution.”