RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

By Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, Matthew Richardson. 2019

858 citations recorded by the source corpus.

When translating natural language questions into SQL queries to answer questions from a database, contemporary semantic parsing models struggle to generalize to unseen database schemas. The generalization challenge lies in (a) encoding the database relations in an accessible way for the semantic parser, and (b) modeling alignment between database columns and their mentions in a given query. We present a unified framework, based on the relation-aware self-attention mechanism, to address schema encoding, schema linking, and feature representation within a text-to-SQL encoder. On the challenging Spider dataset this framework boosts the exact match accuracy to 57.2%, surpassing its best counterparts by 8.7% absolute improvement. Further augmented with BERT, it achieves the new state-of-the-art performance of 65.6% on the Spider leaderboard. In addition, we observe qualitative improvements in the model’s understanding of schema linking and alignment. Our implementation will be open-sourced at https://github.com/Microsoft/rat-sql.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Pre-Trained Language Models for Interactive Decision-Making (2022, methodology): “Successful uses of representations from pre-trained models include syntactic parsing [19] and language-to-code translation [41]; successful adaptations of LM prediction heads include machine translation [45], sentiment classification [5] and style transfer [18].”
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, background): “Different types of structured knowledge, such as databases or knowledge graphs, lead to highly specialized encoders (Lin et al., 2019; Herzig et al., 2020; Wang et al., 2020; Yasunaga et al., 2021).”
Measuring Coding Challenge Competence With APPS (2021, background): “In the past few years, large-scale language models have shown promise in generalizing to various cognitive tasks, including linguistic inference (Wang et al., 2019a), commonsense reasoning (Zellers et al., 2019; Huang et al., 2019; Bisk et al., 2019), logical deduction (Liu et al., 2020b),…”
Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, background): “…learning is applied to tasks such as relation extraction (Wang et al., 2019; Han et al., 2020), natural language generation (Mi et al., 2020), language modelling (Sun et al., 2019) and the pretrained language models adapting to multiple NLP tasks (Wang et al., 2020b; Pfeiffer et al., 2021).”
Learning to Generate Code Sketches (2021, methodology): “We implement P dec ( y j ∣ E ( x t ) , i t , y < j ) with a (causal) relational transformer decoder, similar to Wang et al. [36].”
CodeS: Towards Building Open-source Language Models for Text-to-SQL (2024, neutral): “Once the schema linking results are obtained, for example, the matching degrees for all tables and columns, many techniques [4, 17, 67] incorporate these results as additional input for the text-to-SQL model.”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, background): “There has been work in program synthesis that generates SQL programs for unseen database schemas (Wang et al., 2020; Lin et al., 2020).”
Span-based Semantic Parsing for Compositional Generalization (2020, methodology): “The most dominant approach in recent years for semantic parsing, the task of mapping a natu-ral language utterance to an executable program, has been based on sequence-to-sequence (seq2seq) models (Jia and Liang, 2016; Dong and Lapata, 2016; Wang et al., 2020, inter alia ).”
Few-Shot Semantic Parsing for New Predicates (2021, methodology): “, 2019) and RATSQL (Wang et al., 2019) are two best performing models on the Text-to-SQL benchmark, SPIDER (Yu et al.”
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing (2020, background): “Table 2 shows test results of S M B O P compared to the top (non-anonymous) entries on the leaderboard (Zhao et al., 2021; Shi et al., 2021; Yu et al., 2020; Deng et al., 2020; Lin et al., 2020; Wang et al., 2020a).”
Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions (2021, background): “…community, mapping natural language utterances to SQL queries for databases, also known as SQLbased semantic parsing, has attracted increasing attention (Yin and Neubig, 2017; Dong and Lapata, 2018b; Guo et al., 2019; Wang et al., 2020; Bogin et al., 2019; Lin et al., 2020; Yin and Neubig, 2019).”
LEVER: Learning to Verify Language-to-Code Generation with Execution (2023, methodology): “As L EVER is a simple method that combines few-shot LM generation with learned veriﬁers, it can potentially beneﬁt more advanced prompting methods (Li et al., 2022b; Cheng et al., 2022) or model architectures (Qi et al., 2022; Wang et al., 2020), which we leave as future work.”
Investigating Math Word Problems using Pretrained Multilingual Language Models (2021, background): “This task therefore can be seen as a special case of code synthesis (Desai et al., 2016; Lin et al., 2018; Shin et al., 2019; Wang et al., 2020) in a restricted domain, where the code to be synthesized is the expression tree.”
Mind2Web: Towards a Generalist Agent for the Web (2023, background): “Many studies have centered around environments underpinned by a well-structured schema or ontology, including relational databases [32, 39] and knowledge bases [8, 38], which may not adequately reflect the more heterogeneous conditions in real-world situations.”
Benchmarking Meaning Representations in Neural Semantic Parsing (2020, background): “Grammar-based neural models can guarantee that the generated program is syntactically correct (Yin and Neubig, 2017; Wang et al., 2020; Sun et al., 2020).”
StructGPT: A General Framework for Large Language Model to Reason over Structured Data (2023, methodology): “For Text-to-SQL, we select RAT-SQL+BERT Large (Wang et al., 2020), TKK-Large (Gao et al., 2022a), T5-3B+PICARD (Raffel et al., 2020), RASAT+PICARD (Qi et al., 2022), and RESDSQL-3B+NatSQL (Li et al., 2023a).”
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021, methodology): “…inspired by the schema linking techniques proposed for semantic parsing with structured inputs, such as text to SQL tasks (Iyer et al., 2017; Wang et al., 2019a; Guo et al., 2019), P LOT C ODER ’s encoder connects the embedding of the natural language descriptions with their corresponding…”