Learning Adaptive Language Interfaces through Decomposition

By Siddharth Karamcheti, Dorsa Sadigh, Percy Liang. 2020

32 citations recorded by the source corpus.

Our goal is to create an interactive natural language interface that efficiently and reliably learns from users to complete tasks in simulated robotics settings. We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition: users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps that it can understand. Unfortunately, existing methods either rely on grammars which parse sentences with limited flexibility, or neural sequence-to-sequence models that do not learn efficiently or reliably from individual examples. Our approach bridges this gap, demonstrating the flexibility of modern neural systems, as well as the one-shot reliable generalization of grammar-based methods. Our crowdsourced interactive experiments suggest that over time, users complete complex tasks more efficiently while using our system by leveraging what they just taught. At the same time, getting users to trust the system enough to be incentivized to teach high-level utterances is still an ongoing challenge. We end with a discussion of some of the obstacles we need to overcome to fully realize the potential of the interactive paradigm.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

LILA: Language-Informed Latent Actions (2021, background): “To do this, adding linguistically diverse data has been the gold standard [44, 51]; however, a new class of approaches have emerged that sidestep additional data requirements by tapping into the potential of pretrained language models [43, 50].”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, background): “Furthermore, one should integrate a neural semantic parsing system that helps to interactively learn a joint symbolic language between the machine and the human user through decomposition [16].”
Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring (2021, background): “, through a few turns of interactions with humans (Karamcheti et al., 2020).”
ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, background): “For instance, in embodied AI, the houses may be used for navigation [58, 89, 118, 132, 117, 128, 74, 130], multi-agent interaction [51, 52, 2], rearrangement and interaction [115, 36, 39, 23, 106], manipulation [33, 86, 32, 122], Sim2Real transfer [27, 54, 66], embodied vision-andlanguage [105, 87, ”
In-IDE Code Generation from Natural Language: Promise and Challenges (2021, background): “In addition, in the NLP community there have been notable advancements recently in interactive semantic parsing [51, 119], i.”
ELLA: Exploration through Learned Language Abstraction (2021, background): “Simplified environments also make investigating concepts such as language abstraction possible, and these concepts can generalize to real human-AI interactions (Wang et al., 2017; Karamcheti et al., 2020).”