An analysis of patch plausibility and correctness for generate-and-validate patch generation systems

By Zichao Qi, Fan Long, Sara Achour, Martin C. Rinard. 2015

488 citations recorded by the source corpus.

We analyze reported patches for three existing generate-and- validate patch generation systems (GenProg, RSRepair, and AE). The basic principle behind generate-and-validate systems is to accept only plausible patches that produce correct outputs for all inputs in the validation test suite. Because of errors in the patch evaluation infrastructure, the majority of the reported patches are not plausible — they do not produce correct outputs even for the inputs in the validation test suite. The overwhelming majority of the reported patches are not correct and are equivalent to a single modification that simply deletes functionality. Observed negative effects include the introduction of security vulnerabilities and the elimination of desirable functionality. We also present Kali, a generate-and-validate patch generation system that only deletes functionality. Working with a simpler and more effectively focused search space, Kali generates at least as many correct patches as prior GenProg, RSRepair, and AE systems. Kali also generates at least as many patches that produce correct outputs for the inputs in the validation test suite as the three prior systems. We also discuss the patches produced by ClearView, a generate-and-validate binary hot patching system that lever- ages learned invariants to produce patches that enable systems to survive otherwise fatal defects and security attacks. Our analysis indicates that ClearView successfully patches 9 of the 10 security vulnerabilities used to evaluate the system. At least 4 of these patches are correct.

Subfields: Software, Program Analysis & Verification, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Software

  1. Lingming Zhang
  2. Chun Xia
  3. Charles Sutton
  4. Michele Tufano
  5. Miltiadis Allamanis
  6. Baishakhi Ray
  7. Vincent J. Hellendoorn
  8. Gabriele Bavota

In Program Analysis & Verification

  1. Miltiadis Allamanis
  2. Charles Sutton
  3. Prem Devanbu
  4. Chun Xia
  5. Lingming Zhang
  6. Kensen Shi
  7. Earl T. Barr
  8. Baishakhi Ray

In Signal Processing

  1. Prem Devanbu
  2. Baishakhi Ray
  3. Miltiadis Allamanis
  4. Saikat Chakraborty
  5. Charles Sutton
  6. Vincent J. Hellendoorn
  7. Abhik Roychoudhury
  8. Yangruibo Ding

Citations in the corpus