NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction

By Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Hassan Awadallah. 2021

53 citations recorded by the source corpus.

We study semantic parsing in an interactive setting in which users correct errors with natural language feedback. We present NL-EDIT, a model for interpreting natural language feedback in the interaction context to generate a sequence of edits that can be applied to the initial parse to correct its errors. We show that NL-EDIT can boost the accuracy of existing text-to-SQL parsers by up to 20% with only one turn of correction. We analyze the limitations of the model and discuss directions for improvement and evaluation. The code and datasets used in this paper are publicly available at http://aka.ms/NLEdit.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Self-Refine: Iterative Refinement with Self-Feedback (2023, background): “Humans have been an effective source of feedback (Tandon et al., 2021; Elgohary et al., 2021; Tandon et al., 2022; Bai et al., 2022a).”
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2022, background): “For instance, Elgohary et al. (2021) proposed a model to understand natu-ral language feedback and produce a series of edits to correct a text-to-SQL semantic parser.”
The rise and potential of large language model based agents: a survey (2025, unknown): “Text feedback is usually oﬀered in natural language: humans provide advice on how to modify outputs generated by agents, and the agents then incorporate these suggestions to reﬁne their subsequent outputs [357, 358].”
Constrained Language Models Yield Few-Shot Semantic Parsers (2021, methodology): “…of using the grammar as a constraint , with an eye towards enabling bootstrapping through human-in-the-loop semantic parsing, where humans quickly annotate data by manually correcting parses from an initial prototype (Duan et al., 2016; He et al., 2016; Yao et al., 2019; Elgohary et al., 2021).”
Interscript: A dataset for interactive learning of scripts through error feedback (2021, methodology): “For example, (Elgohary et al. 2021) presented NLE DIT to ﬁx SQL queries using human feed-back such as: replace course id with program id. .”
Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback (2021, unknown): “The corrector model G is trained to operationalize that advice in different ways for different examples appropriately, in contrast to (say) NLEdit where the user-provided speciﬁc corrective edits for a single example only.”