Collaborating with language models for embodied reasoning

By Ishita Dasgupta, Christine Kaeser-Chen, Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, Rob Fergus. 2023

91 citations recorded by the source corpus.

Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. While some sophisticated RL agents can successfully solve difficult tasks, they require a large amount of training data and often struggle to generalize to new unseen environments and new tasks. On the other hand, Large Scale Language Models (LSLMs) have exhibited strong reasoning ability and the ability to to adapt to new tasks through in-context learning. However, LSLMs do not inherently have the ability to interrogate or intervene on the environment. In this work, we investigate how to combine these complementary abilities in a single system consisting of three parts: a Planner, an Actor, and a Reporter. The Planner is a pre-trained language model that can issue commands to a simple embodied agent (the Actor), while the Reporter communicates with the Planner to inform its next command. We present a set of tasks that require reasoning, test this system's ability to generalize zero-shot and investigate failure cases, and demonstrate how components of this system can be trained with reinforcement-learning to improve performance.

Subfields: Artificial Intelligence, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In LLM Agents & Tool Use

Citations in the corpus

Augmented Language Models: a Survey (2023, background): “Dasgupta et al. (2023) use 7B and 70B Chinchilla as planners for an agent that acts and observes the result in a PycoLab environment.”
Large Language Model based Multi-Agents: A Survey of Progress and Challenges (2024, background): “[Dasgupta et al. , 2023] first explores the potential to use LLM as an action planner for embedded agents.”
Language Models can Solve Computer Tasks (2023, background): “WebGPT [42] and WebShop [77] show that LLMs can automate some web-based tasks by introducing a handful of custom commands such as Search <query> and Next Page .”
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents (2023, methodology): “Task planning with LLMs There have been some methods leveraging the large language model to generate action plans for high-level tasks in embodied environments [42, 9, 11].”
Cognitive Architectures for Language Agents (2023, unknown): “Early agents interfaced the LLM directly with the external environment, using it to produce high-level instructions based on the agent’s state (Ahn et al., 2022; Huang et al., 2022c; Dasgupta et al., 2022).”
A survey on large language model based autonomous agents (2023, unknown): “Robotics & Embodied Artificial Intelligence: Recent works have developed more efficient reinforcement learning agents for robotics and embodied artificial intelligence [25, 160, 106, 143, 133, 161, 60, 142, 154, 28, 2].”
Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning (2023, background): “Expanding on this, Dasgupta et al. [2022] added a reporter observing the environment and reporting useful information to the planner.”
Foundation models in robotics: Applications, challenges, and the future (2023, unknown): “Dasgupta et al. [195] combine these complementary abilities in a single system consisting of three parts: a Planner, an Actor, and a Reporter.”
The rise and potential of large language model based agents: a survey (2025, unknown): “Recent studies have indicated that leveraging the rich internal knowledge acquired during the pre-training of LLMs can eﬀectively alleviate these issues [101,171,178,253].”