Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs

By Yu Su, Ahmed Hassan Awadallah, Miaosen Wang, Ryen W. White. 2018

32 citations recorded by the source corpus.

The rapidly increasing ubiquity of computing puts a great demand on next-generation human-machine interfaces. Natural language interfaces, exemplified by virtual assistants like Apple Siri and Microsoft Cortana, are widely believed to be a promising direction. However, current natural language interfaces provide users with little help in case of incorrect interpretation of user commands. We hypothesize that the support of fine-grained user interaction can greatly improve the usability of natural language interfaces. In the specific setting of natural language interface to web APIs, we conduct a systematic study to verify our hypothesis. To facilitate this study, we propose a novel modular sequence-to-sequence model to create interactive natural language interfaces. By decomposing the complex prediction process of a typical sequence-to-sequence model into small, highly-specialized prediction units called modules, it becomes straightforward to explain the model prediction to the user, and solicit user feedback to correct possible prediction errors at a fine-grained level. We test our hypothesis by comparing an interactive natural language interface with its non-interactive version through both simulation and human subject experiments with real-world APIs. We show that with the interactive natural language interface, users can achieve a higher success rate and a lower task completion time, which lead to greatly improved user satisfaction.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, background): “Various supervised classification tasks on webpages have been proposed, including predicting web elements [39], generating API calls [46, 47, 54] and semantic parsing into concept-level navigation actions [31].”
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021, background): “Unlike correction accuracy, Progress is more aligned with user experience in an interactive environment (Su et al., 2018) as it as-”
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study (2019, background): “…semantic parsing has a wide range of applications, including question answering (Berant et al., 2013; Dong and Lapata, 2016; Finegan-Dollak et al., 2018), robot navigation (Artzi and Zettlemoyer, 2013; Thomason et al., 2015) and Web API calling (Quirk et al., 2015; Su et al., 2018).”
FLIN: A Flexible Natural Language Interface for Web Navigation (2020, unknown): “…includes learning tasks from demonstrations (Allen et al., 2007), building reinforcement learning agents (Shi et al., 2017b; Liu et al., 2018b), semantic parsing using seq2seq models to map NL commands into web APIs (Su et al., 2017, 2018) and generating task ﬂows from APIs (Williams et al., 2019).”
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020, methodology): “We use the Spider dataset (Yu et al., 2018) as our source of questions.”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “[88] have proven that end-users preferred a parser system based on an interactive manner over the non-interactive counterpart for NLP interfaces to web APIs.”