A statistical semantic language model for source code

By Tung Thanh Nguyen, Anh Tuan Nguyen, Hoan Anh Nguyen, Tien Nhut Nguyen. 2013

233 citations recorded by the source corpus.

Subfields: Information Systems, Software, Natural Language Processing, Code Generation & Program Synthesis, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

In Natural Language Processing

In Code Generation & Program Synthesis

In Signal Processing

Citations in the corpus

Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation (T) (2015, neutral): “…this study: MeCab to tokenize Japanese sentences [29], pialign to train word alignments [22], KenLM to train Kneser-Ney smoothed language model [30], Moses to train and generate target sentences using the PBMT model [31], and Travatar to train and generate target sentences using T2SMT models [32].”
IntelliCode compose: code generation using transformer (2020, background): “Some of the early applications of machine learning of source code include code search [1, 2], bug detection and localization [3], program synthesis [4], code summarization [5] and code completion [6–10].”
code2vec: learning distributed representations of code (2018, methodology): “…Context-Free Grammars [Allamanis and Sutton 2014; Bielik et al. 2016; Gvero and Kuncak 2015; Maddison and Tarlow 2014], n-grams [Allamanis et al. 2014; Allamanis and Sutton 2013; Hindle et al. 2012; Nguyen et al. 2013; Raychev et al. 2014] have been used for programming languages in the past.”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, methodology): “We choose these languages because they have diﬀerences that could aﬀect the performance of LMs. Java has extensively been used in related work [6, 26, 41, 44, 68, 81].”
Learning programs from noisy data (2016, background): “Existing systems are typically hard-wired to work with a specific prediction strategy and are based on a simple token completion via the n-gram language model [16], on tokens with added annotations [25], other API calls [24, 30], expressions [14], or tree substitutions in the AST [2].”
Bimodal Modelling of Source Code and Natural Language (2015, background): “Hindle et al. (2012); Nguyen et al. (2013); Allamanis & Sutton (2013); Tu et al. (2014) have argued that even simple n-gram-based models can improve over traditional code autocompletion systems.”
Are deep neural networks the best choice for modeling source code? (2017, background): “As we shall see, performance in this final setting is best-in-class, and has ramifications for code synthesis [14, 30], fault localization [8, 31], code naturalization [2] and many real-world code suggestions settings [15, 29].”
Exploring API Embedding for API Usages and Applications (2017, methodology): “Our prior work SLAMC [40] provides a code representation for our phrase-based SMT models in mppSMT [41] and semSMT [42].”
Divide-and-Conquer Approach for Multi-phase Statistical Migration for Source Code (T) (2015, unknown): “First, mppSMT treats a program as a sequence of syntactic units and maps/translates such sequences in two languages to one another.”
Suggesting accurate method and class names (2015, methodology): “Hindle et al. and Ngyuen et al. [25, 41] used n -gram models to improve code autocompletion.”
Extracting code from programming tutorial videos (2016, methodology): “[12] use a statistical language model enriched with semantic information.”
Controlling Conditional Language Models without Catastrophic Forgetting (2021, background): “Controllable code generation Generating source code is an established application of language models (Nguyen et al., 2013; Raychev et al., 2014; Karpathy et al., 2015; Bielik et al., 2016) that since recently has enjoyed renewed interest (Lu et al.”
Multi-task Learning based Pre-trained Language Model for Code Completion (2020, background): “[19] found that source code contained predictable statistical properties, statistical language models began to be used formodeling source code [17, 27, 30, 38, 50], where N-gram is the most widely used model.”
Learning to Fuzz: Application-Independent Fuzz Testing with Probabilistic, Generative Models of Input Data (2016, methodology): “Furthermore, these models can express properties learned by n-gram-based models [22, 38], PCFGs, and conditioning function-based models [10, 41], as well as properties that cannot be expressed with existing approaches.”
Maybe Deep Neural Networks are the Best Choice for Modeling Source Code (2019, methodology): “There is now a large literature on language modeling for code [6, 12, 22, 37, 40, 58, 69].”