Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing

By Daya Guo, Duyu Tang, Nan Duan, M. Zhou, Jian Yin. 2019

50 citations recorded by the source corpus.

In this paper, we present an approach to incorporate retrieved datapoints as supporting evidence for context-dependent semantic parsing, such as generating source code conditioned on the class environment. Our approach naturally combines a retrieval model and a meta-learner, where the former learns to find similar datapoints from the training data, and the latter considers retrieved datapoints as a pseudo task for fast adaptation. Specifically, our retriever is a context-aware encoder-decoder model with a latent variable which takes context environment into consideration, and our meta-learner learns to utilize retrieved datapoints in a model-agnostic meta-learning paradigm for fast adaptation. We conduct experiments on CONCODE and CSQA datasets, where the context refers to class environment in JAVA codes and conversational history, respectively. We use sequence-to-action model as the base semantic parser, which performs the state-of-the-art accuracy on both datasets. Results show that both the context-aware retriever and the meta-learning strategy improve accuracy, and our approach performs better than retrieve-and-edit baselines.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nan Duan
2. Daya Guo
3. Ming Zhou
4. Shujie Liu
5. Duyu Tang
6. Colin B. Clement
7. Mohit Bansal
8. Neel Sundaresan

In Information Systems

1. Nan Duan
2. Daya Guo
3. Neel Sundaresan
4. Alexey Svyatkovskiy
5. Colin B. Clement
6. Shuai Lu
7. Shao Kun Deng
8. Dawn Drain

In Natural Language Processing

1. Nan Duan
2. Neel Sundaresan
3. Ming Zhou
4. Alexey Svyatkovskiy
5. Shao Kun Deng
6. Michele Tufano
7. Shujie Liu
8. Daya Guo

In Meta & Continual Learning

1. Lizhen Qu
2. Gholamreza Haffari
3. Zhuang Li

Citations in the corpus

• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, methodology): “• code-code (clone detection [65, 84, 46, 80, 9, 89, 86], defect detection [91, 55, 51, 42, 78, 79], cloze test [15], code completion [57, 1, 56, 66, 67, 8, 27, 7], code repair [69, 4, 23, 26, 71, 73], and code-to-code translation [48, 37, 10, 41]) • text-code (natural language code search [32, 20, ”
• CodeBLEU: a Method for Automatic Evaluation of Code Synthesis (2020, methodology): “1 (Guo et al. 2019); 2 Fine-tune with GPT-2 (Radford et al. 2019); 3 Pre-trained GPT-2 with the Java data of Codesearchnet (Husain et al. 2019) and then fine-tuning; 4 Fine-tune with CodeBERT (Feng et al. 2020).”
• Context Dependent Semantic Parsing: A Survey (2020, methodology): “T RAN X (Yin and Neubig, 2018) defines the grammars in Abstract Syntax Description Language, while IRN ET (Guo et al., 2019b) considers the context-free grammar of a language called SemQL.”
• Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning (2021, background): “…texts from a certain database (such as the training set) has been widely applied in tasks such as neural machine translation (Gu et al., 2018; Zhang et al., 2018; Xu et al., 2020), text generation (Weston et al., 2018; Kim et al., 2020), semantic parsing (Hashimoto et al., 2018; Guo et al., 2019).”