On Learning Meaningful Code Changes Via Neural Machine Translation
By Michele Tufano, Jevgenija Pantiuchina, Cody Watson, Gabriele Bavota, Denys Poshyvanyk. 2019
249 citations recorded by the source corpus.
Recent years have seen the rise of Deep Learning (DL) techniques applied to source code. Researchers have exploited DL to automate several development and maintenance tasks, such as writing commit messages, generating comments and detecting vulnerabilities among others. One of the long lasting dreams of applying DL to source code is the possibility to automate non-trivial coding activities. While some steps in this direction have been taken (e.g., learning how to fix bugs), there is still a glaring lack of empirical evidence on the types of code changes that can be learned and automatically applied by DL. Our goal is to make this first important step by quantitatively and qualitatively investigating the ability of a Neural Machine Translation (NMT) model to learn how to automatically apply code changes implemented by developers during pull requests. We train and experiment with the NMT model on a set of 236k pairs of code components before and after the implementation of the changes provided in the pull requests. We show that, when applied in a narrow enough context (i.e., small/medium-sized pairs of methods before/after the pull request changes), NMT can automatically replicate the changes implemented by developers during pull requests in up to 36% of the cases. Moreover, our qualitative analysis shows that the model is capable of learning and replicating a wide variety of meaningful code changes, especially refactorings and bug-fixing activities. Our results pave the way for novel research in the area of DL on code, such as the automatic learning and applications of refactoring.
Subfields: Artificial Intelligence, Information Systems, Machine Translation, Code Generation & Program Synthesis, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Charles Sutton
- Rafael-Michael Karampatsis
- Romain Robbes
- Hlib Babii
- Andrea Janes
- Miltiadis Allamanis
- Georgios Gousios
- Michael Pradel
In Information Systems
- Charles Sutton
- Michael Pradel
- Rafael-Michael Karampatsis
- Miltiadis Allamanis
- Georgios Gousios
- Tien Nhut Nguyen
- Romain Robbes
- Hlib Babii
In Machine Translation
In Code Generation & Program Synthesis
- Michael Pradel
- Charles Sutton
- Koushik Sen
- Miltiadis Allamanis
- Rafael-Michael Karampatsis
- Romain Robbes
- Hlib Babii
- Andrea Janes
In Signal Processing
- Michael Pradel
- Charles Sutton
- Koushik Sen
- Rafael-Michael Karampatsis
- Baishakhi Ray
- Miltiadis Allamanis
- Georgios Gousios
- Saikat Chakraborty
Citations in the corpus
- TypeWriter: neural type prediction with search-based validation (2019, background): “Beyond type prediction, others have proposed learning-based techniques to find programming errors [25, 27], predict variable and method names [1, 28, 33], suggest how to improve names [19], search code [10, 30], detect clones [36, 40], classify code [24, 39], predict code edits [31, 38, 41], predict”
- An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation (2018, background): “Some examples of recent models that can be used in a number of software engineering tasks include: code completion [68, 92], defect prediction [85], bug localization [40], clone detection [81, 91], code search [24], learning API sequences [25], recommending method names [3], learning code changes [8”
- Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, background): “Other applications include program repair [20], learning code changes [82], or generating source code comments [47].”
- DLFix: Context-based Code Transformation Learning for Automated Program Repair (2020, methodology): “Tufano et al. [39] learn code changes using sequence-ta-sequence NMT with code abstractions and keyword replacing.”
- Can we generate shellcodes via natural language? An empirical study (2021, background): “[80] investigated the ability of the NMT to learn how to automatically apply code changes implemented by developers during pull requests.”
- Modeling Vocabulary for Big Code Machine Learning (2019, methodology): “, replacing identifiers with placeholders [67], as this prevents recovering the original identifiers.”
- CODIT: Code Editing With Tree-Based Neural Models (2018, methodology): “Researchers have also proposed to automate generic changes by learning either from example edits [20], [21] or from similar patches applied previously to source code [2], [3], [19], [22].”
- When deep learning met code search (2019, background): “ting work has introduced neural networks to identify software defects [10], guide program synthesis[5,35],enablenewrepresentationsforprogramanalysis[1,3],facilitatecodereuse[17],andautomatecodechanges[34].Themodels wepresentheremakeuseofthesetechnologiestovaryingdegreesto explore the design space an”
- On Learning Meaningful Assert Statements for Unit Test Cases (2020, background): “cs.SE] 19 Feb 2020 ICSE ’20, May 23–29, 2020, Seoul, Republic of Korea Cody Watson, Michele Tufano, Kevin Moran, Gabriele Bavota, and Denys Poshyvanyk such as bug-fixing [7, 12, 20, 36], code changes [35], code migration [21, 22], code summarization [18, 19, 39], pseudo-code generation [23], code de”
- RepairAgent: An Autonomous, LLM-Based Agent for Program Repair (2024, unknown): “…the challenge of automatically fixing bugs [1], including techniques based on manually designed [2], [3] and (semi-)automatically extracted [4], [5], [6] fix patterns, based on symbolic constraints [7], [8], [9], and various machine learning-based approaches [10], [11], [12], [13], [14], [15], [16].”