Are deep neural networks the best choice for modeling source code?

By Vincent J. Hellendoorn, Premkumar T. Devanbu. 2017

346 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Code Generation & Program Synthesis

Citations in the corpus

The adverse effects of code duplication in machine learning models of code (2018, methodology): “Language models have been used for autocompletion [14, 15, 20, 24] and it would be unreasonable to assume that the true distribution of this particular use cases contains duplicate code.”
A systematic evaluation of large language models of code (2022, background): “Recently, LMs have shown impressive performance in modeling also source code, written in programming languages (Hindle et al., 2016; Hellendoorn & Devanbu, 2017; Alon et al., 2020; Karampatsis et al., 2020).”
Unified Pre-training for Program Understanding and Generation (2021, neutral): “Code Summarization (MovshovitzAttias and Cohen, 2013; Allamanis et al., 2016; Iyer et al., 2016; Alon et al., 2019a; Hu et al., 2018; Harer et al., 2019; Ahmad et al., 2020), Bug Detection (Ray et al., 2016; Li et al., 2018b; Russell et al., 2018; Zhou et al., 2019; Chakraborty et al., 2020),…”
TypeWriter: neural type prediction with search-based validation (2019, background): “The out-of-vocabulary problem is well know in software engineering research [16].”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, neutral): “models, suggesting that the traditional models may achieve comparable results or even outperform deep models [178].”
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair (2018, background): “commits that focus on a clear task; 2) contrary to natural language, misuse of rare words (identifiers, numbers, etc) is often fatal in programming languages [7]; in natural”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, methodology): “Tu et al. [81] and Hellendoorn and Devanbu [41] use n -gram caches.”
Learning to Fix Build Errors with Graph2Diff Neural Networks (2019, background): “ficult problem [21]. Using pointers, the machine learning component can predict both where and how to fix. Finally, thecopy mechanism addresses the well-known out-of-vocabulary problem of source code [5, 16, 17]: source code projects often include projectspecific identifiers that do not occur in the”
Improved Code Summarization via a Graph Neural Network (2020, background): “But evidence is accumulating that the metaphor to NMT has major limits [21].”
Code Prediction by Feeding Trees to Transformers (2020, background): “We present MRR as a percentage, in keeping with prior work [7], [32].”
RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems (2023, background): “The inception of Language Models (LMs) in code completion can be traced back to the usage of n-gram based LMs [37, 20], RNN models [41], and probabilistic grammar-models [7, 32, 19], which laid the foundation for the subsequent introduction of more advanced LMs in this field.”
Learning-Based Recursive Aggregation of Abstract Syntax Trees for Code Clone Detection (2019, methodology): “Others are trained with graph embedding algorithms [13], derived from training token-level Recurrent Neural Networks as language models for code [13], [17], [25], [26].”
Typilus: neural type hints (2020, background): “Code completion [32, 34, 36, 52] is one of the most widely explored topics.”
Modeling Vocabulary for Big Code Machine Learning (2019, methodology): “observe that in general, source code is more repetitive than natural language [33], Hellendoorn and Devanbu notice that NLMs trained on a software corpus would struggle due to vocabulary size [32].”
Few-shot training LLMs for project-specific code-summarization (2022, background): “Vocabularies of a project are highly local, and functions from the same projects are likely to share same set of identifiers [10, 22].”