Controllable Semantic Parsing via Retrieval Augmentation

By Panupong Pasupat, Yuan Zhang, Kelvin Guu. 2021

50 citations recorded by the source corpus.

In practical applications of semantic parsing, we often want to rapidly change the behavior of the parser, such as enabling it to handle queries in a new domain, or changing its predictions on certain targeted queries. While we can introduce new training examples exhibiting the target behavior, a mechanism for enacting such behavior changes without expensive model re-training would be preferable. To this end, we propose ControllAble Semantic Parser via Exemplar Retrieval (CASPER). Given an input query, the parser retrieves related exemplars from a retrieval index, augments them to the query, and then applies a generative seq2seq model to produce an output parse. The exemplars act as a control mechanism over the generic generative model: by manipulating the retrieval index or how the augmented query is constructed, we can manipulate the behavior of the parser. On the MTOP dataset, in addition to achieving state-of-the-art on the standard setup, we show that CASPER can parse queries in a new domain, adapt the prediction toward the specified patterns, or adapt to new semantic schemas without having to further re-train the model.

Subfields: Artificial Intelligence, Molecular Biology, Retrieval-Augmented Generation, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Luke S. Zettlemoyer
2. Jonathan Berant
3. Hannaneh Hajishirzi
4. Percy Liang
5. Danqi Chen
6. Mohit Bansal
7. Noah A. Smith
8. Yejin Choi

In Molecular Biology

1. Jonathan Herzig
2. Jonathan Berant
3. Ohad Rubin
4. Noah A. Smith
5. Jiebo Luo
6. Zhengyuan Yang
7. Fernando Diaz
8. Michael Bendersky

In Retrieval-Augmented Generation

1. Danqi Chen
2. Luyu Gao
3. Akari Asai
4. Jonathan Berant
5. Sewon Min
6. Zexuan Zhong
7. Hamed Zamani
8. Jonathan Herzig

In Information Extraction

1. Jonathan Herzig
2. Panupong Pasupat
3. Jonathan Berant
4. Ohad Rubin
5. Kelvin Guu
6. Noah A. Smith
7. Ni Lao
8. Jiebo Luo

Citations in the corpus

• Learning To Retrieve Prompts for In-Context Learning (2021, methodology): “Similar to us, Pasupat et al. (2021) proposed to use retrieval in semantic parsing.”
• Retrieval-based Language Models and Applications (2023, methodology): “We then discuss methods designed to build more powerful retrieval-based LMs for certain downstream tasks, such as dialogue (Shus-ter et al., 2021), semantic parsing (Pasupat et al., 2021), and machine translation (Khandelwal et al., 2021; Zheng et al., 2021).”
• PromptCap: Prompt-Guided Task-Aware Image Captioning (2022, background): “Prompting has been successful for a variety of natural language tasks [37], including but not limited to classification tasks [42, 56], semantic parsing [44, 74], knowledge generation [35, 57], and dialogue systems [20, 28].”
• Attributed Text Generation via Post-hoc Research and Revision (2022, background): “, 2020), semantic parsing (Pasupat et al., 2021), or tasks requiring creativity), illustrating that end-task performance and attribution are not always aligned.”
• Retrieval-Enhanced Machine Learning (2022, methodology): “[44] proposed using exemplar retrieval for semantic parsing.”
• Few-Shot Semantic Parsing with Language Models Trained on Code (2021, background): “Pasupat et al. (2021) proposed a retrievalaugmented solution to semantic parsing, which relates to the dynamic prompt selection of Shin et al. (2021), and which we followed here without alteration.”