Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets

By Annibale Panichella, Fitsum Meshesha Kifetew, Paolo Tonella. IEEE Transactions on Software Engineering, 2018

346 citations recorded by the source corpus.

The test case generation is intrinsically a multi-objective problem, since the goal is covering multiple test targets (e.g., branches). Existing search-based approaches either consider one target at a time or aggregate all targets into a single fitness function (whole-suite approach). Multi and many-objective optimisation algorithms (MOAs) have never been applied to this problem, because existing algorithms do not scale to the number of coverage objectives that are typically found in real-world software. In addition, the final goal for MOAs is to find alternative trade-off solutions in the objective space, while in test generation the interesting solutions are only those test cases covering one or more uncovered targets. In this paper, we present Dynamic Many-Objective Sorting Algorithm (DynaMOSA), a novel many-objective solver specifically designed to address the test case generation problem in the context of coverage testing. DynaMOSA extends our previous many-objective technique Many-Objective Sorting Algorithm (MOSA) with dynamic selection of the coverage targets based on the control dependency hierarchy. Such extension makes the approach more effective and efficient in case of limited search budget. We carried out an empirical study on 346 Java classes using three coverage criteria (i.e., statement, branch, and strong mutation coverage) to assess the performance of DynaMOSA with respect to the whole-suite approach (WS), its archive-based variant (WSA) and MOSA. The results show that DynaMOSA outperforms WSA in 28 percent of the classes for branch coverage (+8 percent more coverage on average) and in 27 percent of the classes for mutation coverage (+11 percent more killed mutants on average). It outperforms WS in 51 percent of the classes for statement coverage, leading to +11 percent more coverage on average. Moreover, DynaMOSA outperforms its predecessor MOSA for all the three coverage criteria in 19 percent of the classes with +8 percent more code coverage on average.

Subfields: Computer Science Applications, Software

View Paper: https://www.semanticscholar.org/paper/0943d656221144bf063a538f6c667ac006c303cd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Software

Citations in the corpus

An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation (2023, methodology): “Their results show that, on most of their target 27 Python projects, their proposed technique, C ODA M OSA , outperforms the baseline search-based technique, Pynguin’s implementation of M OSA [64], as well as using only Codex.”
sFuzz: An Efficient Adaptive Fuzzer for Solidity Smart Contracts (2020, background): “A known problem for such a strategy [27] is that the number of seeds in the same Pareto frontier soars with the increase of the number of objectives (i.”
CodeT: Code Generation with Generated Tests (2022, background): “, 2015), DynaMOSA (Panichella et al., 2017), and MIO (Arcuri, 2017), were proposed to automatically generate test cases for statically typed programming languages like Java.”
Automated Unit Test Generation for Python (2020, methodology): “generation, which provide ample opportunities for further research, for example, the integration of further test-generation algorithms, such as (Dyna)MOSA [16], the generation of assertions, or the integration of type inference approaches.”
Pynguin: Automated Unit Test Generation for Python (2022, methodology): “Furthermore, DynaMOSA yields the highest coverage values.”