Improving Natural Language Interaction with Robots Using Advice
By Nikhil Mehta, Dan Goldwasser. 2019
16 citations recorded by the source corpus.
Over the last few years, there has been growing interest in learning models for physically grounded language understanding tasks, such as the popular blocks world domain. These works typically view this problem as a single-step process, in which a human operator gives an instruction and an automated agent is evaluated on its ability to execute it. In this paper we take the first step towards increasing the bandwidth of this interaction, and suggest a protocol for including advice, high-level observations about the task, which can help constrain the agent’s prediction. We evaluate our approach on the blocks world task, and show that even simple advice can help lead to significant performance improvements. To help reduce the effort involved in supplying the advice, we also explore model self-generated advice which can still improve results.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Robotics & Control
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yiming Yang
- Peter Clark
- Aman Madaan
- Niket Tandon
- Keisuke Sakaguchi
- Hannaneh Hajishirzi
- Ashish Sabharwal
- Mohit Bansal
In Computer Vision and Pattern Recognition
- Niket Tandon
- Aman Madaan
- Keisuke Sakaguchi
- Yiming Yang
- Peter Clark
- Luc Van Gool
- Yi Yang
- Trevor Darrell
In Natural Language Processing
- Peter Clark
- Aman Madaan
- Yiming Yang
- Niket Tandon
- Keisuke Sakaguchi
- Ashish Sabharwal
- Yejin Choi
- Mohit Bansal
Citations in the corpus
- The rise and potential of large language model based agents: a survey (2025, unknown): “Text feedback is usually offered in natural language: humans provide advice on how to modify outputs generated by agents, and the agents then incorporate these suggestions to refine their subsequent outputs [357, 358].”
- Interscript: A dataset for interactive learning of scripts through error feedback (2021, background): “In language-based interactions, controlled settings (Mehta and Goldwasser 2019; Wang, Liang, and Manning 2016) give a better handle and are easy to evaluate.”
- Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback (2021, unknown): “(3) Providing hints: (Mehta and Goldwasser, 2019) show how a system can learns to understand regional (e.”