Learning Programmatic Idioms for Scalable Semantic Parsing

By Srini Iyer, Alvin Cheung, Luke S. Zettlemoyer. 2019

48 citations recorded by the source corpus.

Programmers typically organize executable source code using high-level coding patterns or idiomatic structures such as nested loops, exception handlers and recursive blocks, rather than as individual code tokens. In contrast, state of the art (SOTA) semantic parsers still map natural language instructions to source code by building the code syntax tree one node at a time. In this paper, we introduce an iterative method to extract code idioms from large source code corpora by repeatedly collapsing most-frequent depth-2 subtrees of their syntax trees, and train semantic parsers to apply these idioms during decoding. Applying idiom-based decoding on a recent context-dependent semantic parsing task improves the SOTA by 2.2% BLEU score while reducing training time by more than 50%. This improved speed enables us to scale up the model by training on an extended training set that is 5\times larger, to further move up the SOTA by an additional 2.3% BLEU and 0.9% exact match. Finally, idioms also significantly improve accuracy of semantic parsing to SQL on the ATIS-SQL dataset, when training data is limited.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Uri Alon
2. Eran Yahav
3. Nan Duan
4. Omer Levy
5. Graham Neubig
6. Mohit Bansal
7. Jianfeng Gao
8. Luke S. Zettlemoyer

In Information Systems

1. Uri Alon
2. Eran Yahav
3. Miltiadis Allamanis
4. Omer Levy
5. Nan Duan
6. Daya Guo
7. Roy Sadaka
8. Neel Sundaresan

In Natural Language Processing

1. Uri Alon
2. Eran Yahav
3. Omer Levy
4. Nan Duan
5. Graham Neubig
6. Miltiadis Allamanis
7. Roy Sadaka
8. Marc Brockschmidt

In Computational Linguistics & Semantics

1. Ge Li
2. Mitchell Stern
3. Lizhen Qu
4. Gholamreza Haffari
5. Daniel Klein
6. Omer Levy
7. Zhuang Li

Citations in the corpus

• Unified Pre-training for Program Understanding and Generation (2021, neutral): “Generation Iyer et al. (2018) NL to Java 100,000 2,000 2,000”
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, background): “For example, the idiom mining task [2, 35] is to extract code idioms, which are syntactic fragments that recur across software projects and serve a single semantic purpose [2].”
• Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, methodology): “Similar to (Shin et al., 2019; Iyer et al., 2019), the base parser converts the utterance x into a se-”
• Structural Language Models of Code (2019, methodology): “Node Trees vs. Production Trees While we predict a single node at each step, previous work (Iyer et al., 2018; 2019) predicts a grammar production rule.”
• Few-Shot Semantic Parsing for New Predicates (2021, methodology): “Thus we apply a template normalization procedure in a similar manner as (Iyer et al., 2019) to pre-process all LF templates.”
• Semantic Scaffolds for Pseudocode-to-Code Generation (2020, background): “, 2016) that do not account for structure on the output side, or tree-based models (Allamanis et al., 2015; Rabinovich et al., 2017a; Yin and Neubig, 2017; Hayati et al., 2018; Iyer et al., 2019) that incorporate the syntax but not the semantics of the output domain.”
• Program Synthesis and Semantic Parsing with Learned Code Idioms (2019, background): “Concurrently with this work, Iyer et al. [17] developed a different approach of learning code idioms for semantic parsing.”