Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

By Ziyu Yao, Yu Su, Huan Sun, Wen-tau Yih. 2019

87 citations recorded by the source corpus.

As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results. In this paper, we propose a new, unified formulation of the interactive semantic parsing problem, where the goal is to design a model-based intelligent agent. The agent maintains its own state as the current predicted semantic parse, decides whether and where human intervention is needed, and generates a clarification question in natural language. A key part of the agent is a world model: it takes a percept (either an initial question or subsequent feedback from the user) and transitions to a new state. We then propose a simple yet remarkably effective instantiation of our framework, demonstrated on two text-to-SQL datasets (WikiSQL and Spider) with different state-of-the-art base semantic parsers. Compared to an existing interactive semantic parsing approach that treats the base parser as a black box, our approach solicits less user feedback but yields higher run-time accuracy.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers (2021, background): “Feedback semantic parsing interactively collects data from user feedbacks as continuous data streams but does not address the problem of catastrophic forgetting or improve forward transfer (Iyer et al., 2017; Yao et al., 2019; Labutov et al., 2018).”
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021, background): “question posed by the system (Yao et al., 2019; Gur et al., 2018) or suggestions of edits that can be applied to the parse (Su et al.”
Constrained Language Models Yield Few-Shot Semantic Parsers (2021, methodology): “…of using the grammar as a constraint , with an eye towards enabling bootstrapping through human-in-the-loop semantic parsing, where humans quickly annotate data by manually correcting parses from an initial prototype (Duan et al., 2016; He et al., 2016; Yao et al., 2019; Elgohary et al., 2021).”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “[85] develop a model-based interactive semantic parsing (MISP) as the general principle for interactive semantic parsing.”
Photon: A Robust Cross-Domain Text-to-SQL System (2020, background): “These include semantic parsing (Dong and Lapata, 2018; Zhang et al., 2019), ambiguity detection and conﬁdence estimation (Dong et al., 2018; Yao et al., 2019), natural language response generation (Liu et al., 2019) and so on.”
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020, background): “Yao et al. (2019b) ask yes/no questions about the presence of SQL components while generating a SQL parse one component at a time.”
Context Dependent Semantic Parsing: A Survey (2020, methodology): “Most Feedback Semantic Parsing systems (Iyer et al., 2017; Yao et al., 2019b; Yao et al., 2019a; Elgohary et al., 2020) start with using an C I SP parser to parse a given utterance into an initial MR .”
Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions (2021, background): “The rules guarantee that the suggested question matches the SQL query (Yao et al., 2019).”
Learning Adaptive Language Interfaces through Decomposition (2020, background): “Yao et al. (2019) defined a similar conversational system for Text-to-SQL models that decides when intervention is needed, and generates a clarification question accordingly.”
The rise and potential of large language model based agents: a survey (2025, unknown): “Furthermore, agents can learn human preference from comparative scores like multiple choice [355,356].”
Teaching Large Language Models to Self-Debug (2023, background): “, 2023a), interact with users for multi-turn code generation (Yu et al., 2019; Yao et al., 2019; Nijkamp et al., 2023), and fix generated programs based on human feedback (Iyer et al.”