Towards a theory of natural language interfaces to databases

By Ana-Maria Popescu, Oren Etzioni, Henry A. Kautz. 2003

664 citations recorded by the source corpus.

The need for Natural Language Interfaces to databases (NLIs) has become increasingly acute as more and more people access information through their web browsers, PDAs, and cell phones. Yet NLIs are only usable if they map natural language questions to SQL queries correctly. As Schneiderman and Norman have argued, people are unwilling to trade reliable and predictable user interfaces for intelligent but unreliable ones. In this paper, we introduce a theoretical framework for reliable NLIs, which is the foundation for the fully implemented Precise NLI. We prove that, for a broad class of semantically tractable natural language questions, Precise is guaranteed to map each question to the corresponding SQL query. We report on experiments testing Precise on several hundred questions drawn from user studies over three benchmark databases. We find that over 80% of the questions are semantically tractable questions, which Precise answers correctly. Precise automatically recognizes the 20% of questions that it cannot handle, and requests a paraphrase. Finally, we show that Precise compares favorably with Mooney's learning NLI and with Microsoft's English Query product

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semantic Parsing on Freebase from Question-Answer Pairs (2013, methodology): “Besides semantic parsing for querying databases (Popescu et al., 2003; Clarke et al., 2010; Liang et al., 2011), previous work has looked at interpreting natural language for performing programming tasks (Kushman and Barzilay, 2013; Lei et al.”
Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning (2017, background): “CISE [Popescu et al., 2003], which translates questions to SQL queries and identifies questions that it is not confident about.”
Learning Dependency-Based Compositional Semantics (2011, methodology): “• Precise (Popescu et al., 2003) does not use learning, but instead relies on matching words to strings in the database using various heuristics based on WordNet and the Charniak parser.”
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (2018, background): “Other existing text-to-SQL datasets also include Restau-rants (Tang and Mooney, 2001b; Popescu et al., 2003a), Scholar (Iyer et al., 2017), Academic (Li and Jagadish, 2014), Yelp and IMDB (Yagh-mazadeh et al., 2017b), Advising (Finegan-Dollak et al., 2018), and WikiSQL (Zhong et al., 2017).”
Compositional Semantic Parsing on Semi-Structured Tables (2015, background): “, 2011) usually cover a smaller scope of operations and compositionality, while early statistical semantic parsers for question answering (Wong and Mooney, 2007; Zettlemoyer and Collins, 2007) and high-accuracy natural language interfaces for databases (Androutsopoulos et al., 1995; Popescu et al., ”
Language to Logical Form with Neural Attention (2016, methodology): “Model hyper-parameters were cross-validated Method Accuracy COCKTAIL (Tang and Mooney, 2001) 79.4 PRECISE (Popescu et al., 2003) 88.0 ZC05 (Zettlemoyer and Collins, 2005) 79.3 DCS+L (Liang et al., 2013) 90.7 TISP (Zhao and Huang, 2015) 85 on the training set for J OBS and G EO .”
Learning a Neural Semantic Parser from User Feedback (2017, methodology): “A few systems have been developed to directly generate SQL queries from natural language (Popescu et al., 2003; Giordani and Mos-chitti, 2012; Poon, 2013).”
NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System (2018, background): “The dataset we introduce provides a new type of target meaning representations (Bash 1 commands), and is signiﬁcantly larger (from two to ten times) than most existing semantic parsing benchmarks (Dahl et al., 1994; Popescu et al., 2003).”
Constructing an Interactive Natural Language Interface for Relational Databases (2014, methodology): “Aspointed out in [13], the major usability problem in NLIDBs is its limited reliability.”
CodeS: Towards Building Open-source Language Models for Text-to-SQL (2024, positive): “The training portion encompasses 7,000 manually annotated samples, supplemented by an additional 1,659 samples sourced from six previous text-to-SQL datasets: Restaurants [50, 63], GeoQuery [80], Scholar [30], Academic [35], IMDB, and Yelp [72].”
On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries (2020, neutral): “Widely-used domain-speciﬁc datasets include ATIS (Price, 1990; Dahl et al., 1994), GeoQuery (Zelle and Mooney, 1996; Popescu et al., 2003), Restaurants (Tang and Mooney, 2000; Popescu et al., 2003), and Scholar (Iyer et al., 2017).”
DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction (2023, background): “…research in this ﬁeld has progressed through several phases, with early systems being domain-speciﬁc, supporting controlled natural language (Popescu et al., 2003, 2004; Li et al., 2007; Li and Jagadish, 2014) or relying on rule-based approaches (Stratica et al., 2005) while more recent…”
SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning (2017, background): “The study of translating natural language into SQL queries has a long history (Warren & Pereira, 1982; Androutsopoulos et al., 1993; 1995; Popescu et al., 2003; 2004; Li et al., 2006; Giordani & Moschitti, 2012; Zhang & Sun, 2013; Li & Jagadish, 2014; Wang et al., 2017).”
Translating Questions to SQL Queries with Generative Parsers Discriminatively Reranked (2012, methodology): “PRECISE (Popescu et al., 2003) is the only system evaluated on GEO700 in terms of correct SQL queries.”
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (2019, neutral): “(Warren and Pereira, 1982; Zettlemoyer and Collins, 2005; Popescu et al., 2003; Li et al., 2006; Li and Jagadish, 2014; Iyer et al., 2017; Zhong et al., 2017; Xu et al., 2017; Yu et al., 2018a; Dong and Lapata, 2018; FineganDollak et al., 2018; Guo et al., 2019; Bogin et al., 2019).”
SQLizer: query synthesis from natural language (2017, background): “As a result, there has been a flurry of interest in automatically synthesizing SQL queries from informal specifications [Feng et al. 2017a; Li and Jagadish 2014; Popescu et al. 2003; Wang et al. 2017; Zhang and Sun 2013].”
SpreadsheetCoder: Formula Prediction from Semi-structured Context (2021, background): “…several new challenges compared to related work in programming by example (Gulwani, 2011; Bunel et al., 2018; Balog et al., 2017), semantic parsing (Popescu et al., 2003; Zhong et al., 2017; Yu et al., 2018) and source code completion (Ray-chev et al., 2014; Li et al., 2018; Svyatkovskiy et al.,…”