Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

By Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch. 2022

1,645 citations recorded by the source corpus.

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g."make breakfast"), to a chosen set of actionable steps (e.g."open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Reinforcement Learning

Citations in the corpus

ReAct: Synergizing Reasoning and Acting in Language Models (2022, neutral): “Perhaps most relevant to ReAct in this respect are SayCan (Ahn et al., 2022) and Inner Monologue (Huang et al., 2022b), which use LLMs for robotic action planning and decision making.”
Emergent Abilities of Large Language Models (2022, neutral): “The TruthfulQA benchmark (Lin et al., 2021) showed that GPT-3 models were more likely to mimic human falsehoods as they got larger, though Rae et al. (2021) later showed on a multiple-choice version that scaling Gopher to 280B enabled emergent performance substantially better than random.”
GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, background): “Given an appropriate prompt (such as a story of a character working to achieve a goal), LLMs can predict and thus simulate an agent (Huang et al., 2022).”
Augmented Language Models: a Survey (2023, background): “Similarly, Huang et al. (2022a) investigate whether it is possible to use the world knowledge captured by LMs to take specific actions in response to high-level tasks written in natural language such as “make breakfast”.”
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, background): “The latter in effect compares to [18], which loses the explicit option probabilities, and thus is less interpretable and cannot be combined with affordance probabilities.”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, methodology): “Similar to ours are recent task planning approaches that leverage pre-trained autoregressive LLMs to decompose abstract, high-level instructions into a sequence of low-level steps executable by an agent [20, 21] in a zero-shot manner.”
Pre-Trained Language Models for Interactive Decision-Making (2022, background): “In contrast to (Huang et al., 2022), which investigates the existing knowledge in pre-trained LMs by prompting, our work focuses on adapting/finetuning pre-trained LMs for decision-making tasks using imitation learning.”
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “…analogies to 3D vision and robotics, this can be thought of as building an on-thefly reconstruction of the sequence of events in the observable world with language, rather than other representations, such as dynamically-updated 3D meshes (Izadi et al. 2011) or neural fields (Tancik et al. 2022).”
PaLM-E: An Embodied Multimodal Language Model (2023, methodology): “…LLM’s generation and an eligible set of instructions (Huang et al., 2022b), incorporating affordance functions (Ahn et al., 2022), visual feedback (Huang et al., 2022c), generating world models (Nottingham et al., 2023; Zellers et al., 2021a), planning over graphs and maps (Shah et al., 2022;…”
Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models (2023, methodology): “Typically, these prior methods use pretrained models to imagine visual [16, 2] or textual plans [8, 28, 29, 39], which then inform a low-level robot control policy.”
Visual Language Maps for Robot Navigation (2022, methodology): “In contrast to prior work, which may reference these subgoals with language and map to low-level policies with semantic translation [39] or affordances [35], [40], [41], [42], we leverage the code-writing capabilities of LLMs to generate executable Python robot code [43], [33], [44], [27] that can (”
Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling (2023, methodology): “Existing work that uses LLMs for generating action plans focuses on methods for grounding language in environment states (Ichter et al., 2022; Huang et al., 2022b; Song et al., 2022b), or improving LLM plans through more structured output (Singh et al., 2022; Liang et al., 2022b).”
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change (2022, background): “These roles range from generating plans [11, 33] and heuristics [33, 1] to extracting planning knowledge [7].”
WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “Large Language Models (LLMs) have been observed to break down high-level assignments into more manageable sub-tasks [11].”