Code completion with statistical language models

By Veselin Raychev, Martin T. Vechev, Eran Yahav. 2014

722 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Software, Natural Language Processing, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Software

In Natural Language Processing

In Code Generation & Program Synthesis

Citations in the corpus

Competition-level code generation with AlphaCode (2022, background): “Therefore, most prior work has been limited to either restricted domain-specific programming languages (Gulwani, 2011) or short code snippets (Bruch et al., 2009; Raychev et al., 2014).”
Program Synthesis with Large Language Models (2021, methodology): “Early work applied statistical n-gram models [Hindle et al., 2012, Allamanis and Sutton, 2013a] and neural networks [Maddison and Tarlow, 2014, Raychev et al., 2014] to code.”
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “Code completion systems based on language models that generate code tokens [3, 34, 49, 57], rather than natural-language tokens, are the basis of intelligent IDEs [11] such as Deep TabNine [16] and Microsoft’s Visual Studio IntelliCode [45].”
The adverse effects of code duplication in machine learning models of code (2018, methodology): “Language models have been used for autocompletion [14, 15, 20, 24] and it would be unreasonable to assume that the true distribution of this particular use cases contains duplicate code.”
A systematic evaluation of large language models of code (2022, methodology): “These models excel at useful downstream tasks like code completion (Raychev et al., 2014) and synthesizing code from natural language descriptions (Desai et al.”
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “Similarly, developers who are confused about what to write next can use code completion systems [57, 1, 56, 66, 67, 8, 27, 7] to automatically complete the following tokens based on the edits made to the code.”
PyMT5: Multi-mode Translation of Natural Language and Python Code with Transformers (2020, methodology): “…many NLP tasks have practical utility in software development environments: language modeling or generation can be used for code completion (Raychev et al., 2014; Bruch et al., 2009; Svyatkovskiy et al., 2019, 2020), translation/summarization to generate documentation or natural language…”
code2seq: Generating Sequences from Structured Representations of Code (2018, background): “Several papers model code as a sequence of tokens (Iyer et al., 2016; Allamanis et al., 2016; Loyola et al., 2017), characters (Bielik et al., 2017), and API calls (Raychev et al., 2014).”
Learning to Represent Programs with Graphs (2017, background): “Currently, existing deep learning models of source code capture its shallow, textual structure, e.g. as a sequence of tokens (Hindle et al., 2012; Raychev et al., 2014; Allamanis et al., 2016), as parse trees (Maddison & Tarlow, 2014; Bielik et al., 2016), or as a ﬂat dependency networks of…”
IntelliCode compose: code generation using transformer (2020, background): “Numerous intelligent code completion systems for both statically and dynamically typed languages have been proposed [6, 11, 41, 42].”
An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation (2018, methodology): “To this aim, we employ a different decoding strategy called Beam Search, used in previous applications of deep learning [6, 9, 21, 67].”
DeepBugs: a learning approach to name-based bug detection (2018, neutral): “Existing approaches address a variety of development tasks, including code completion based on probabilistic models of code [Bielik et al. 2016; Raychev et al. 2016a, 2014], predicting fixes of compilation errors via neural networks [Bhatia and Singh 2016; Gupta et al. 2017], and the generation of…”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, methodology): “Precise scoring of code fragments is essential for tasks such as translating a program from one programming language to another [54, 66], code completion [32, 76], and code synthesis from natural language and vice versa [7, 27, 67, 73].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “Machine learning can aid programming and software engineering in various ways: automatic code completion (Raychev et al., 2014; Svyatkovskiy et al., 2019), program synthesis (Bunel et al., 2018; Kulal et al., 2019), program repair (Vasic et al., 2019; Yasunaga and Liang, 2020), code search (Husain…”
Controlling Conditional Language Models without Catastrophic Forgetting (2021, background): “Controllable code generation Generating source code is an established application of language models (Nguyen et al., 2013; Raychev et al., 2014; Karpathy et al., 2015; Bielik et al., 2016) that since recently has enjoyed renewed interest (Lu et al.”
On Learning Meaningful Code Changes Via Neural Machine Translation (2019, methodology): “To this aim, we employ a decoding strategy called a Beam Search used in previous applications of DL [57].”
Software Defect Prediction via Convolutional Neural Network (2017, methodology): “[41] employed Recurrent Neural Network (RNN) to tackle code completion task, which was further improved by their later Decision Trees-based method [42].”