Code as Policies: Language Model Programs for Embodied Control

By Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Peter R. Florence, Andy Zeng. 2022

1,688 citations recorded by the source corpus.

Large language models (LLMs) trained on code-completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g., from object detectors [2], [3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions (‘faster’) depending on context (i.e., behavioral commonsense). This paper presents Code as Policies: a robot-centric formulation of language model generated programs (LMPs) that can represent reactive policies (e.g., impedance controllers), as well as waypoint-based policies (vision-based pick and place, trajectory-based control), demonstrated across multiple real robot platforms. Central to our approach is prompting hierarchical code-gen (recursively defining undefined functions), which can write more complex code and also improves state-of-the-art to solve 39.8% of problems on the HumanEval [1] benchmark. Code and videos are available at https://code-as-policies.github.io

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, LLM Agents & Tool Use, Code Generation & Program Synthesis, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Mohit Bansal
  3. Jianfeng Gao
  4. Yoshua Bengio
  5. Quoc V. Le
  6. Chelsea Finn
  7. Percy Liang
  8. Pieter Abbeel

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Jitendra Malik
  6. Andrew Zisserman
  7. Ming-Hsuan Yang
  8. Li Fei-Fei

In Control and Systems Engineering

  1. Sergey Levine
  2. Chelsea Finn
  3. Pieter Abbeel
  4. Li Fei-Fei
  5. Yuke Zhu
  6. Pierre Sermanet
  7. Vikash Kumar
  8. Nicolas Manfred Otto Heess

In LLM Agents & Tool Use

  1. Brian Ichter
  2. Andy Zeng
  3. Cordelia Schmid
  4. Chelsea Finn
  5. Corey Lynch
  6. Tianmin Shu
  7. Pierre Sermanet
  8. Dorsa Sadigh

In Code Generation & Program Synthesis

  1. Frank F. Xu
  2. Graham Neubig
  3. Shuyan Zhou
  4. Josh Tenenbaum
  5. Hao Peng
  6. Prithviraj Ammanabrolu
  7. Hannaneh Hajishirzi
  8. Alane Suhr

In Robotics & Control

  1. Sergey Levine
  2. Chelsea Finn
  3. Li Fei-Fei
  4. Pieter Abbeel
  5. Corey Lynch
  6. Yuke Zhu
  7. Pierre Sermanet
  8. Nicolas Manfred Otto Heess

Citations in the corpus