Building a Semantic Parser Overnight

By Yushi Wang, Jonathan Berant, Percy Liang. 2015

376 citations recorded by the source corpus.

How do we build a semantic parser in a new domain starting with zero training examples? We introduce a new methodology for this setting: First, we use a simple grammar to generate logical forms paired with canonical utterances. The logical forms are meant to cover the desired set of compositional operators, and the canonical utterances are meant to capture the meaning of the logical forms (although clumsily). We then use crowdsourcing to paraphrase these canonical utterances into natural utterances. The resulting data is used to train the semantic parser. We further study the role of compositionality in the resulting paraphrases. Finally, we test our methodology on seven domains and show that we can build an adequate semantic parser in just a few hours.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “, 2020), as well as from synthetic data to real data (including sim2real) (Wang et al., 2015; Ganin & Lempitsky, 2015; Richter et al., 2016; Peng et al., 2018; Hellendoorn et al., 2019; Xu et al., 2020; Bellemare et al., 2020), as synthetic data can be easier to obtain than real data.”
Analysing Mathematical Reasoning Abilities of Neural Models (2019, methodology): “ies/GCSE/1211-GCSE-UnitUMS-Boundaries(Science-Mathematics).pdf 9 Published as a conference paper at ICLR 2019 another would be to obtain more question templates via mechanical turking, as proposed by Wang et al. (2015). Finally one completely distinct direction the dataset could be extended is to in”
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning (2017, methodology): “The datasets are GeoQuery880 (Tang & Mooney, 2001), ATIS (Price, 1990), Free917 (Cai & Yates, 2013), Overnight (Wang et al., 2015), WebQues-tions (Berant et al., 2013), and WikiTableQues-tions (Pasupat & Liang, 2015).”
Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models (2022, methodology): “The logic-to-text task is ONR (OverNightReverse), which is constructed based on the textto-logic benchmark Overnight (Wang et al., 2015) (detailed in Appendix B).”
The Web as a Knowledge-Base for Answering Complex Questions (2018, methodology): “Thus, we sample question-query pairs, automatically create more complex SPARQL queries, generate automatically questions that are understandable to AMT workers, and then have them paraphrase those into nat-ural language (similar to Wang et al. (2015)).”
Neural Programmer: Inducing Latent Programs with Gradient Descent (2015, background): “In a recent work by Wang et al. (2015) to build semantic parsers for 7 domains, the authors hand engineer a separate grammar for each domain.”
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (2016, background): “This is reminiscent of the floating parser (Wang et al., 2015; Pasupat and Liang, 2015), where a deriva-”
Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, methodology): “We select Overnight (Wang et al., 2015b) and NLMapV2 (Lawrence and Riezler, 2018) to simulate the proposed three continual learning scenarios, coined OVERNIGHT, NLMAP(QT) and NLMAP(CITY), respectively.”
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow (2018, background): “de-block sized candidates, aiming to recover code solutions that exactly answer the intent. Finally, some work has asked programmers to manually write NL descriptions for code [20, 30], or vice-versa [41]. This allows for the generation of high-quality data, but is time consuming and does not scale ”
Learning a Neural Semantic Parser from User Feedback (2017, background): “, 2016)), learning from paraphrases (often gathered through crowdsourcing (Wang et al., 2015)), data augmentation (e.”
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data (2019, methodology): “Other approaches to semi-automatic dataset creation also use paraphrasing (Wang et al., 2015; Su et al., 2016).”
Learning executable semantic parsers for natural language understanding (2016, background): “One idea is to use general para-phrasing models to map input utterances to the “canonical utterances” of logical forms [5, 30].”
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships (2018, background): “Finally, crowdsourcing datasets has become a driving force in AI, producing signiﬁcant progress, e.g., (Rajpurkar et al. 2016; Joshi et al. 2017; Wang, Berant, and Liang 2015).”
Natural Language to Structured Query Generation via Meta-Learning (2018, background): “…studied in natu-ral language processing research (Zettlemoyer and Collins, 2005; Giordani and Moschitti, 2010; Artzi and Zettlemoyer, 2011; Berant et al., 2013; Vla-chos and Clark, 2014; Yih et al., 2014, 2015; Wang et al., 2015; Golub and He, 2016; Iyer et al., 2017; Krishnamurthy et al., 2017).”
Question Asking as Program Generation (2017, background): “Machine systems for question answering have also fruitfully modeled questions as programs [24, 12], and computational work in cognitive science has modeled various kinds of concepts as programs [17, 5, 13].”
Practical Semantic Parsing for Spoken Language Understanding (2019, methodology): “For Q&A, we consider Overnight (Wang et al., 2015b) and NLmaps (Lawrence and Riezler, 2016).”