Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners
By Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jacob Varley and 4 more. Conference on Robot Learning, 2023
392 citations recorded by the source corpus.
Large language models (LLMs) exhibit a wide range of promising capabilities -- from step-by-step planning to commonsense reasoning -- that may provide utility for robots, but remain prone to confidently hallucinated predictions. In this work, we present KnowNo, which is a framework for measuring and aligning the uncertainty of LLM-based planners such that they know when they don't know and ask for help when needed. KnowNo builds on the theory of conformal prediction to provide statistical guarantees on task completion while minimizing human help in complex multi-step planning settings. Experiments across a variety of simulated and real robot setups that involve tasks with different modes of ambiguity (e.g., from spatial to numeric uncertainties, from human preferences to Winograd schemas) show that KnowNo performs favorably over modern baselines (which may involve ensembles or extensive prompt tuning) in terms of improving efficiency and autonomy, while providing formal assurances. KnowNo can be used with LLMs out of the box without model-finetuning, and suggests a promising lightweight approach to modeling uncertainty that can complement and scale with the growing capabilities of foundation models. Website: https://robot-help.github.io
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/d1500f1dbd62e26ef0753f31e845078f58479968
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shunyu Yao
- Theodore R. Sumers
- Karthik Narasimhan
- Thomas L. Griffiths
- Sergey Levine
- Jacob Andreas
- Mohit Bansal
- Matthew J. Hausknecht
Citations in the corpus
- Cognitive Architectures for Language Agents (2023, background): “Agents capable of generating language may ask for help (Ren et al., 2023; Nguyen et al., 2022b; 2019; Nguyen and Daumé III, 2019) or clarification (Biyik and Palan, 2019; Sadigh et al., 2017; Padmakumar et al., 2022; Thomason et al., 2020; Narayan-Chen et al., 2019) – or entertain or emotionally…”
- Foundation models in robotics: Applications, challenges, and the future (2023, neutral): “5) Case Study: Uncertainty Quantification for Language-Instructed Robots: Recently, there has been exciting progress in performing rigorous uncertainty quantification for language-instructed robots [216].”