Neural Attribute Machines for Program Generation

By Matthew Amodio, Swarat Chaudhuri, Thomas W. Reps. 2017

35 citations recorded by the source corpus.

Recurrent neural networks have achieved remarkable success at generating sequences with complex structures, thanks to advances that include richer embeddings of input and cures for vanishing gradients. Trained only on sequences from a known grammar, though, they can still struggle to learn rules and constraints of the grammar. Neural Attribute Machines (NAMs) are equipped with a logical machine that represents the underlying grammar, which is used to teach the constraints to the neural machine by (i) augmenting the input sequence, and (ii) optimizing a custom loss function. Unlike traditional RNNs, NAMs are exposed to the grammar, as well as samples from the language of the grammar. During generation, NAMs make significantly fewer violations of the constraints of the underlying grammar than RNNs trained only on samples from the language of the grammar.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Code Generation & Program Synthesis

Citations in the corpus

A Survey of Machine Learning for Big Code and Naturalness (2017, background): “[16] create a significantly more complex model that aims to learn to enforce deterministic constraints of the code generation rather than enforcing them on the directly on the output.”
Unsupervised Translation of Programming Languages (2020, background): “To address this issue, several approaches proposed to use additional constraints on the decoder, to ensure that the generated functions respect the syntax of the target language [3, 4, 5, 40, 48].”
code2vec: learning distributed representations of code (2018, background): “…2018; Raychev et al. 2015], code completion [Mishne et al. 2012; Raychev et al. 2014], code summarization [Allamanis et al. 2016], code generation [Amodio et al. 2017; Lu et al. 2017; Maddison and Tarlow 2014; Murali et al. 2017], and more (see [Allamanis et al. 2017; Vechev and Yahav 2016] for a…”
DeepBugs: a learning approach to name-based bug detection (2018, methodology): “…predicting fixes of compilation errors via neural networks [Bhatia and Singh 2016; Gupta et al. 2017], and the generation of inputs for fuzz testing via probabilistic, generative language models [Patra and Pradel 2016] or neural networks [Amodio et al. 2017; Godefroid et al. 2017; Liu et al. 2017].”
Generative Code Modeling with Graphs (2018, background): “Finally, Amodio et al. [4] and Yin and Neubig [24] follow a left-to-right, depth-ﬁrst expansion strategy, but thread updates to single state (via a gated recurrent unit) through the overall generation procedure, thus giving the pickProduction procedure access to the full generation history as well…”
Improving bug detection via context-based code representation learning and attention-based neural networks (2019, methodology): “…correction for syntax errors [Bhatia and Singh 2016], fuzz testing with probabilistic, generative models [Patra and Pradel 2016], program synthesis [Amodio et al. 2017], code clones [Li et al. 2017; Smith and Horwitz 2009; White et al. 2016], program classification and summarization [Allamanis et…”
Structural Language Models of Code (2019, background): “Maddison & Tarlow (2014) and Amodio et al. (2017) generate general-purpose unconditional code, and do not deal with the challenge of fitting the code to a given context.”
Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts (2017, background): “to programs, targeting a variety of development tasks such as code completion [10, 29, 32], fixing syntactic errors [9, 18], generating inputs for fuzz testing of programs [17, 24, 28], and generating programs or short code snippets to assist developers or test compilers [4, 34].”
Structural Language Models for Any-Code Generation (2019, background): “Maddison and Tarlow (2014) and Amodio et al. (2017) generate supposedly general-purpose code in a modern programming language, but do not deal with the challenge of ﬁtting the code to a given context.”