PHOG: Probabilistic Model for Code

By Pavol Bielik, Veselin Raychev, Martin T. Vechev. 2016

203 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Code Generation & Program Synthesis

Citations in the corpus

The adverse effects of code duplication in machine learning models of code (2018, methodology): “Although all metrics worsen, the effect differs. For example, JavaScript-150k and Java-Large have very similar (file-level) duplication but the impact of duplication on the evaluation metrics of PHOG [9] and code2vec [6] is quite different. This can be attributed to two factors (a) different models ”
A Survey of Machine Learning for Big Code and Naturalness (2017, methodology): “m) Token NN (Encoder-Decoder) GUI Code Synthesis Bhatia and Singh [25] PD(c) Token RNN (LSTM) Syntax Error Correction Bhoopchand et al. [26] PD(c) Token NN (Pointer Net) Code Completion Bielik et al. [29] PD(c) Syntax PCFG + annotations Code Completion Campbell et al. [35] PD(c) Token n-gram Syntax ”
Neuro-Symbolic Program Synthesis (2016, background): “There is also a recent line of work on learning probabilistic models of code from a large number of code repositories (big code) (Raychev et al., 2015; Bielik et al., 2016; Hindle et al., 2016), which are then used for applications such as auto-completion of partial programs, inference of variable a”
code2seq: Generating Sequences from Structured Representations of Code (2018, background): “Works that use syntactic information when learning code usually use it for relatively easier tasks which mainly focus on “filling gaps” in a given program (Alon et al., 2018b; Bielik et al., 2016; Raychev et al., 2016, 2015; Allamanis et al., 2018) or semantic classification of code snippets (Alon e”
Learning to Represent Programs with Graphs (2017, background): “…models of source code capture its shallow, textual structure, e.g. as a sequence of tokens (Hindle et al., 2012; Raychev et al., 2014; Allamanis et al., 2016), as parse trees (Maddison & Tarlow, 2014; Bielik et al., 2016), or as a ﬂat dependency networks of variables (Raychev et al., 2015).”
code2vec: learning distributed representations of code (2018, background): “However, as observed previously [Anonymous 2018; Bielik et al. 2016; Raychev et al. 2015], programming languages can greatly benefit from representations that leverage the structured nature of their syntax.”
DeepBugs: a learning approach to name-based bug detection (2018, methodology): “ address it via supervised learning. To effectively address this task, our approach relies on training data for both classes, i.e., examples of both correct and incorrect code. As observed by others [Bielik et al. 2016; Nguyen and Nguyen 2015; Raychev et al. 2015], the huge amount of existing code p”
Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code (2020, methodology): “Other models of source code include probabilistic higher order grammars (PHOG) [14], which use ASTs, and several types of RNNs, including LSTMs [26, 41, 88].”
Probabilistic model for code with decision trees (2016, background): “As discussed throughout the paper, in terms of expressiveness, the recent works of [7, 27] are perhaps most advanced and precise: they condition not only on a fixed context but also on a dynamically learned context.”
SmartPaste: Learning to Adapt Source Code (2017, unknown): “Currently, existing machine learning models of source code capture its shallow, textual structure, e.g. as a sequence of tokens [3, 10], as parse trees [7, 11], or as a ﬂat dependency networks of variables [16].”
Language-Agnostic Representation Learning of Source Code from Structure and Context (2021, background): “Several works leverage structured graphical models for probabilistic models of source code, usually through parse trees (Maddison & Tarlow, 2014; Bielik et al., 2016).”
Neural Code Comprehension: A Learnable Representation of Code Semantics (2018, methodology): “As for representing the context of a token, which is necessary for training embeddings, some works rely on lexicographical locality [2, 18, 20], whereas others exploit the structural nature of code, using Data Flow Graphs [4], Control Flow Graphs [51, 53, 64], Abstract Syntax Trees (ASTs) [12, 30], ”
Structured Neural Summarization (2018, background): “The use of additional structure on related tasks on source code has been studied recently, for example in models that are conditioned on learned traversals of the syntax tree (Bielik et al., 2016) and in graph-based approaches (Allamanis et al., 2018; Cvitkovic et al., 2018).”
Are deep neural networks the best choice for modeling source code? (2017, background): “code, suppressing the information content of source code to around 1 bit per token! Notably, unlike [6, 24, 29, 33] our approach is language-agnostic.”
Learning to Fuzz: Application-Independent Fuzz Testing with Probabilistic, Generative Models of Input Data (2016, background): “PHOG [10] and Deep3 [41] learn a model to predict how to complete existing data, e.”
Maybe Deep Neural Networks are the Best Choice for Modeling Source Code (2019, background): “This stage is necessary to be able to run the implementation on any dataset other than the one used in the original paper [12].”