InferFix: End-to-End Program Repair with LLMs

By Ma Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, Alexey Svyatkovskiy. ESEC/SIGSOFT FSE, 2023

324 citations recorded by the source corpus.

Software development life cycle is profoundly influenced by bugs; their introduction, identification, and eventual resolution account for a significant portion of software development cost. This has motivated software engineering researchers and practitioners to propose different approaches for automating the identification and repair of software defects. Large Language Models (LLMs) have been adapted to the program repair task through few-shot demonstration learning and instruction prompting, treating this as an infilling task. However, these models have only focused on learning general bug-fixing patterns for uncategorized bugs mined from public repositories. In this paper, we propose : a transformer-based program repair framework paired with a state-of-the-art static analyzer to fix critical security and performance bugs. combines a Retriever – transformer encoder model pretrained via contrastive learning objective, which aims at searching for semantically equivalent bugs and corresponding fixes; and a Generator – an LLM (12 billion parameter Codex Cushman model) finetuned on supervised bug-fix data with prompts augmented via adding bug type annotations and semantically similar fixes retrieved from an external non-parametric memory. To train and evaluate our approach, we curated , a novel, metadata-rich dataset of bugs extracted by executing the Infer static analyzer on the change histories of thousands of Java and C# repositories. Our evaluation demonstrates that outperforms strong LLM baselines, with a top-1 accuracy of 65.6% for generating fixes in C# and 76.8% in Java. We discuss the deployment of alongside Infer at Microsoft which offers an end-to-end solution for detection, classification, and localization of bugs, as well as fixing and validation of candidate patches, integrated in the continuous integration (CI) pipeline to automate the software development workflow.

Subfields: Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/34d24b2d9f116f8f652c112d4ac924afcf11bd0d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

Citations in the corpus

Large Language Models for Software Engineering: Survey and Open Problems (2023, background): “Jin et al. [157] proposed InferFix, which contains a LLM (Codex Cushman) finetuned on supervised bug-fix data.”
Software Testing With Large Language Models: Survey, Landscape, and Vision (2023, unknown): “Jiang et al. [113] check the CodeSearchNet and BigQuery, which are the data sources of common LLMs, and the results show that four repositories used by the Defect4J benchmark are also in CodeSearchNet, and the whole Defects4J repository is included by BigQuery.”
Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024, unknown): “InferFix [131] crafts a prompt carrying the bug type, location, relevant syntax hierarchies, and similar fixes through dense retrieval.”
TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation (2023, background): “Large Language Models (LLMs) have exhibited remarkable proficiency in generating text that closely resembles human language and in performing a wide range of tasks [69], including Natural Language Processing [4, 32, 54], Robotics [10, 43, 56], and Information Retrieval [1, 24, 25, 48].”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “As of now, there are a few works that discuss how to use LLMs to protect data integrity.”
Large Language Models for Software Engineering: A Systematic Literature Review (2023, background): “For example, researchers often take a collected dataset of code snippets and manually annotate them with bug types to create a constructed dataset for studying automated program repair techniques [70, 131, 351].”
AutoCodeRover: Autonomous Program Improvement (2024, unknown): “However, obtaining buggy locations for a large project is an essential and challenging task in resolving real-life bug reports.”
Evaluating Large Language Models in Class-Level Code Generation (2024, methodology): “These include code generation [46, 54, 62, 72], code summarization [12, 51] and various software maintenance tasks, including vulnerability detection [58, 61, 73], test generation [68, 69], and program repair [26, 36, 37, 63].”