Robust Text-to-SQL Generation with Execution-Guided Decoding

By Chenglong Wang, Kedar Tatwawadi, Marc Brockschmidt, Po-Sen Huang, Yi Mao, Oleksandr Polozov, Rishabh Singh. 2018

120 citations recorded by the source corpus.

We consider the problem of neural semantic parsing, which translates natural language questions into executable SQL queries. We introduce a new mechanism, execution guidance, to leverage the semantics of SQL. It detects and excludes faulty programs during the decoding procedure by conditioning on the execution of partially generated program. The mechanism can be used with any autoregressive generative model, which we demonstrate on four state-of-the-art recurrent or template-based semantic parsing models. We demonstrate that execution guidance universally improves model performance on various text-to-SQL datasets with different scales and query complexity: WikiSQL, ATIS, and GeoQuery. As a result, we achieve new state-of-the-art execution accuracy of 83.8% on WikiSQL.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Reasoning Like Program Executors (2022, methodology): “Other lines of research leverage program execution in inference as a reliable sanity guarantee for generated programs by pruning non-executable candidates (Wang et al., 2018b; Chen et al., 2019b, 2021; Odena et al., 2020; Ellis et al., 2019; Chen et al., 2019b; Sun et al., 2018; Zohar and Wolf, 2018”
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers (2019, methodology): “For simplicity of experiments, we did not implement either BERT augmentation or execution-guided decoding (EG) (Wang et al., 2018), both of which are common in state-of-the-art WikiSQL models.”
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data (2020, background): “…been adapted to operate over structured DB tables (Wang et al., 2015; Xu et al., 2017; Dong and Lapata, 2018; Yu et al., 2018b; Shi et al., 2018; Wang et al., 2018), and open-domain, semi-structured Web tables (Pasupat and Liang, 2015; Sun et al., 2016; Neelakantan et al., 2016 2018)), and…”
Execution-Guided Neural Program Synthesis (2018, methodology): “Prior works also consider incorporating syntax constraints and information from program execution to facilitate program synthesis (Devlin et al., 2017b; Wang et al., 2018a; Bunel et al., 2018).”
Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation (2019, background): “…manner and achieve beyond 80% exact matching accuracy on public Text-to-SQL benchmarks (e.g., ATIS, GeoQuery and WikiSQL) (Krishna-murthy et al., 2017; Zhong et al., 2017; Xu et al., 2017; Yaghmazadeh et al., 2017; Yu et al., 2018a; Dong and Lapata, 2018; Wang et al., 2018; Hwang et al., 2019).”
LEGO: Latent Execution-Guided Reasoning for Multi-Hop Question Answering on Knowledge Graphs (2021, methodology): “Our executionguided reasoning framework is related to the line of research on execution-guided program synthesis (Odena et al., 2020; Chen et al., 2019a; Wang et al., 2018; Nye et al., 2020a).”
LEVER: Learning to Verify Language-to-Code Generation with Execution (2023, methodology): “Intermediate execution results were used to guide program search at both training (Chen et al., 2019; 2021b) and inference time (Wang et al., 2018).”
TAPEX: Table Pre-training via Learning a Neural SQL Executor (2021, methodology): “Execution-Guided Decoding is proposed to leverage execution results of SQL queries during inference (Wang et al., 2018).”