Generative Code Modeling with Graphs

By Marc Brockschmidt, Miltiadis Allamanis, Alexander L. Gaunt, Oleksandr Polozov. 2018

193 citations recorded by the source corpus.

Generative models for source code are an interesting structured prediction problem, requiring to reason about both hard syntactic and semantic constraints as well as about natural, likely programs. We present a novel model for this problem that uses a graph to represent the intermediate state of the generated output. The generative procedure interleaves grammar-driven expansion steps with graph augmentation and neural message passing steps. An experimental evaluation shows that our new model can generate semantically meaningful expressions, outperforming a range of strong baselines.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback (2020, background): “, 2018) and more recently, source code (Allamanis et al., 2018; Brockschmidt et al., 2019).”
SPoC: Search-based Pseudocode to Code (2019, background): “In any case, the search space of possible parses can be controlled by validating the constructed partial parses and pruning invalid ones [50, 6].”
A Hazard Analysis Framework for Code Synthesis Large Language Models (2022, background): “Deriving from programming language synthesis techniques may lead the way on how this can be carried out [9, 12, 16, 18, 32].”
code2seq: Generating Sequences from Structured Representations of Code (2018, neutral): “, 2015b), and even generation (Balog et al., 2017; Rabinovich et al., 2017; Yin and Neubig, 2017; Devlin et al., 2017; Murali et al., 2017; Brockschmidt et al., 2019).”
GraphCodeBERT: Pre-training Code Representations with Data Flow (2020, background): “, 2020), code generation (Rabinovich et al., 2017; Yin & Neubig, 2017; Brockschmidt et al., 2018), code clone detection (Wei & Li, 2017; Zhang et al.”
Learning to Fix Build Errors with Graph2Diff Neural Networks (2019, background): “not.override.abstract 3% compiler.err.already.defined 0% compiler.err.missing.ret.stmt Figure 8: Accuracy by kind of the first diagnostic for those that appeared at least 10 times in validation data. [9, 20, 43], in that these models map from a graph to a sequence of decisions that can include selec”
Typilus: neural type hints (2020, methodology): “Inspired by previous work [5, 9, 15, 20, 51], our graph encodes the tokens, syntax tree, data flow, and symbol table of each Python program and can be thought as a form of feature extraction.”
Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks (2020, background): “Recently machine learning has broadened the capabilities of static analysis, offering progress on challenging problems like code completion [5], bug ﬁnding [2, 13], and program repair [25].”
Learning to Prove Theorems via Interacting with Proof Assistants (2019, background): “Similarly, a program can be generated by a deep network as a sequence of tokens (Hindle et al., 2012), an AST (Parisotto et al., 2016) or a graph (Brockschmidt et al., 2019).”
A large-scale benchmark for few-shot program induction and synthesis (2021, methodology): “Machine learning, and deep learning in particular, have been increasingly used to improve the search over programs (Parisotto et al., 2016; Kalyan et al., 2018; Brockschmidt et al., 2018; Ellis et al., 2019; Nye et al., 2020).”
DiGress: Discrete Denoising diffusion for graph generation (2022, background): “These successes raise hope for building powerful models for graph generation, a task with diverse applications such as molecule design (Liu et al., 2018), traffic modeling (Yu & Gu, 2019), and code completion (Brockschmidt et al., 2019).”
Adversarial examples for models of code (2019, background): “els of code against attacks that perturb names, and in particular providing Anti-DAMP defense. •An experimental evaluation of attacking three neural architectures: code2vec [6], GGNN [1], and GNNFiLM [13] in two languages: Java and C#. Our evaluation shows that our adversarial technique can change a”
Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations (2022, background): “, 2021), and even program synthesis (Brockschmidt et al., 2019).”
Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree (2020, methodology): “[24] used GGNN to generate program expressions for code completion in C#.”
Program Synthesis and Semantic Parsing with Learned Code Idioms (2019, methodology): “Prior work [7, 38, 40] also showed that tree-and graph-based code generation models outperform sequence-to-sequence models, and thus we adopt a similar architecture in this work.”
Structural Language Models of Code (2019, methodology): “Metrics Following Brockschmidt et al. (2019), we report exact match accuracy at 1 and 5.”
Code Prediction by Feeding Trees to Transformers (2020, background): “Among other flavors of code prediction are the ones where code after the prediction location, when available, is taken into account [9], [49], [51], [52], e.”
Learning Autocompletion from Real-World Datasets (2020, background): “[10] is composed of 593 popular, opensource C# projects.”