Structural Language Models for Any-Code Generation

By Uri Alon, Roy Sadaka, Omer Levy, Eran Yahav. 2019

19 citations recorded by the source corpus.

We address the problem of Any-Code Generation (AnyGen) - generating code without any restriction on the vocabulary or structure. The state-of-the-art in this problem is the sequence-to-sequence (seq2seq) approach, which treats code as a sequence and does not leverage any structural information. We introduce a new approach to AnyGen that leverages the strict syntax of programming languages to model a code snippet as a tree - structural language modeling (SLM). SLM estimates the probability of the program's abstract syntax tree (AST) by decomposing it into a product of conditional probabilities over its nodes. We present a neural model that computes these conditional probabilities by considering all AST paths leading to a target node. Unlike previous structural techniques that have severely restricted the kinds of expressions that can be generated, our approach can generate arbitrary expressions in any programming language. Our model significantly outperforms both seq2seq and a variety of existing structured approaches in generating Java and C# code. We make our code, datasets, and models available online.

Subfields: Information Systems, Software, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Software

In Natural Language Processing

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, methodology): “They support tasks such as extracting code semantics [2, 4], and code and edit completion [3,10,21,57], with several commercial products adopting these techniques [11, 16].”
Unsupervised Translation of Programming Languages (2020, background): “To address this issue, several approaches proposed to use additional constraints on the decoder, to ensure that the generated functions respect the syntax of the target language [3, 4, 5, 40, 48].”
Code Prediction by Feeding Trees to Transformers (2020, background): “tion, especially recurrent neural networks [26, 28, 29] and graph neural networks [6, 13, 43]. Among other flavors of code completion, such as where program after the predicting location is available [6, 9, 13, 36] or where the granularity of prediction is smaller (e.g. characters [12] or subtokens ”
Adversarial examples for models of code (2019, background): “Neural models of code have recently achieved state-of-theart performance on various tasks such as prediction of variable names and types [1, 5, 10, 32], code summarization [2, 3, 18], code generation [4, 11, 28], code search [13, 25, 34], and bug finding [29, 33, 35].”