AutoQA: From Databases to Q&A Semantic Parsers with Only Synthetic Training Data

By Silei Xu, Sina J. Semnani, Giovanni Campagna, Monica S. Lam. 2020

55 citations recorded by the source corpus.

We propose AutoQA, a methodology and toolkit to generate semantic parsers that answer questions on databases, with no manual effort. Given a database schema and its data, AutoQA automatically generates a large set of high-quality questions for training that covers different database operations. It uses automatic paraphrasing combined with template-based parsing to find alternative expressions of an attribute in different parts of speech. It also uses a novel filtered auto-paraphraser to generate correct paraphrases of entire sentences. We apply AutoQA to the Schema2QA dataset and obtain an average logical form accuracy of 62.9% when tested on natural questions, which is only 6.4% lower than a model trained with expert natural language annotations and paraphrase data collected from crowdworkers. To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset. AutoQA achieves 69.8% answer accuracy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “…Kamath et al., 2020), as well as from synthetic data to real data (including sim2real) (Wang et al., 2015; Ganin & Lempitsky, 2015; Richter et al., 2016; Peng et al., 2018; Hellendoorn et al., 2019; Xu et al., 2020; Bellemare et al., 2020), as synthetic data can be easier to obtain than real data.”
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021, background): “Several other approaches have demonstrated the benefit of adopting this approach to train semantic parsers in low-resource settings (Su et al., 2017; Zhong et al., 2017; Cheng et al., 2018; Xu et al., 2020).”
Effective Distant Supervision for Temporal Relation Extraction (2020, background): “This observation follows a trend of recent work showing pre-trained models’ ability to generalize from synthetic data to natural data (Xu et al., 2020; Marzoev et al., 2020).”
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning (2021, background): “It also includes semantic parsing systems trained either without supervision (Liang et al., 2017; Mou et al., 2017; Muhlgay et al., 2019), or with synthetic language data (Marzoev et al., 2020; Xu et al., 2020b).”
SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models (2022, methodology): “Other ways of bootstrapping a semantic parsing requires rules/grammars to synthesize training examples (Xu et al., 2020; Wang et al., 2015; Yu et al., 2020; Campagna et al., 2019; Weir et al., 2020; Marzoev et al., 2020; Campagna et al., 2020).”
Constrained Language Models Yield Few-Shot Semantic Parsers (2021, methodology): “Xu et al. (2020) In this paper we take the approach of using the grammar as a constraint , with an eye towards enabling bootstrapping through human-in-the-loop semantic parsing, where humans quickly annotate data by manually correcting parses from an initial prototype (Duan et al., 2016; He et al.,…”