Is the cure worse than the disease? overfitting in automated program repair

By Edward K. Smith, Earl T. Barr, Claire Le Goues, Yuriy Brun. ESEC/SIGSOFT FSE, 2015

392 citations recorded by the source corpus.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/c14982de9adbfaa1848052fad254f8d9c5cbb61f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

Citations in the corpus

Automated program repair (2019, background): “More commonly, patches produced by current repair techniques tend to overfit the provided test suite by disabling (or deleting) under-tested functionality [11].”
DLFix: Context-based Code Transformation Learning for Automated Program Repair (2020, methodology): “Then, the test cases and/or programverification are applied to select the better candidate fixes [36].”
Precise Condition Synthesis for Program Repair (2016, background): “However, the precisions of existing repair techniques are not high [10], [12], [13].”
TBar: revisiting template-based automated program repair (2019, background): “However, these patches could be overfitting [27, 58] and even worse than the bug [63].”
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair (2018, background): “The patch may yet be incorrect because of the overfitting problem [32].”
AutoCodeRover: Autonomous Program Improvement (2024, neutral): “Learning-based APR techniques [20, 30, 39] often train a deep learning model with large code repositories and are guided by a specific representation of code syntax and semantics to predict the next tokens that are most likely to be correct patch.”
History Driven Program Repair (2016, background): “Although the risk of low quality patches can be mitigated by using more comprehensive test suites to guide the search process, even with full-coverage test suites, existing test-guided techniques may be susceptible to overfitting [47].”
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation (2018, background): “Further benefits include the static nature of NMT when identifying candidate patches, since, unlike some generate-and-validate approaches, we do not need to execute test cases during patch generation [73, 93].”
Baldur: Whole-Proof Generation and Repair with Large Language Models (2023, background): “Unfortunately, these automated program repair techniques often overfit to the available tests and produce patches that, while passing all the tests, fail to encode the developers’ intent [59, 66, 75, 85].”
Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs (2018, background): “operators do not overfit the inputs of input-output specifications [40].”
The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs (2015, methodology): “The INTROCLASS benchmark was similarly used in another evaluation of Gen- Prog [24], [25].”
Learning to Fix Build Errors with Graph2Diff Neural Networks (2019, background): “d it is unlikely that the predicted fix is what the developer intended to do. This is a example of false positive, and it is known as the overfitting problem in the automated program repair community [32]. +importstaticjunit.framework.Assert.assertFalse;//Groundtruth +importstaticorg.junit.Assert.as”
Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis (2016, background): “Still, such an overfitting repair [38] (an incorrect repair that merely passes the provided tests) can be helpful in debugging, because the user can at least see that the (incorrectly) repaired if conditional may be buggy.”