Language-Conditioned Imitation Learning for Robot Manipulation Tasks

By Simon Stepputtis, Joseph Campbell, Mariano Phielipp, Stefan Lee, Chitta Baral, Heni Ben Amor. 2020

251 citations recorded by the source corpus.

Imitation learning is a popular approach for teaching motor skills to robots. However, most approaches focus on extracting policy parameters from execution traces alone (i.e., motion trajectories and perceptual data). No adequate communication channel exists between the human expert and the robot to describe critical aspects of the task, such as the properties of the target object or the intended shape of the motion. Motivated by insights into the human teaching process, we introduce a method for incorporating unstructured natural language into imitation learning. At training time, the expert can provide demonstrations along with verbal descriptions in order to describe the underlying intent (e.g., "go to the large green bowl"). The training process then interrelates these two modalities to encode the correlations between language, perception, and motion. The resulting language-conditioned visuomotor policies can be conditioned at runtime on new human commands and instructions, which allows for more fine-grained control over the trained policies while also reducing situational ambiguity. We demonstrate in a set of simulation experiments how our approach can learn language-conditioned manipulation policies for a seven-degree-of-freedom robot arm and compare the results to a variety of alternative methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Cross-Task Generalization via Natural Language Crowdsourcing Instructions (2021, background): “Previous work in NLP has studied “instructions” (Goldwasser and Roth, 2014) in different domains such as robotic instructions (Shridhar et al., 2020; Stepputtis et al., 2020), database commands (Kim et al.”
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “Most focus on following low-level instructions [15, 17, 53, 16, 5, 48], though some methods address long-horizon, compound tasks in simulated domains [54, 55, 51].”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “Various works have investigated strategies for the challenging problem of fusing vision, language, and control [60, 61, 62, 63, 64, 65, 66].”
LILA: Language-Informed Latent Actions (2021, background): “Prior works integrate language in robotics settings for similar purposes within the full autonomy paradigm [9, 10, 11, 12, 13].”
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022, background): “…are a versatile way of deﬁn-ing goals, which is why they have been studied in the context of a variety of applications, such as instructions in grounded environments (Shrid-har et al., 2020; Stepputtis et al., 2020; Min et al., 2022b; Weir et al., 2022) and database commands (Kim et al., 2020).”
Language Conditioned Imitation Learning Over Unstructured Data (2020, methodology): “Unlike concurrent IL approaches to robotic instruction following [44], which assume access to labeled task demonstrations and pretrained object detection, our method learns from unstructured and unlabeled demonstration data and learns perception, language understanding, and control fully end-to-end…”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “Many of these works adapt to the new scenario by condi59 tioning on a robot demonstration [1, 2], a video of a human [3, 4], a language instruction [23, 24], 60 or a goal image [21].”
LanCon-Learn: Learning with Language to Enable Generalization in Multi-Task Manipulation (2021, background): “These approaches, as well as more modern research [8], [21], [22], [23], [24], [25], [26], [27], [28], typically require a large corpus of commands or observations with granular instructions, as well as a detailed alignment of commands to states.”
Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models (2022, background): “Recent advances in deep learning with large amounts of data have led to advances in methods for learning instruction-conditioned policies [1, 28, 34, 43, 44].”
R3M: A Universal Visual Representation for Robot Manipulation (2022, background): “Prior works have explored the use of natural language in robot manipulation, primarily as a means of task specification [51, 52, 35, 53] or reward learning [54].”
Benchmarking Generalization via In-Context Instructions on 1, 600+ Language Tasks (2022, background): “…are a versatile way of deﬁning goals, which is why they have been studied in the context of a variety of applications, such as instructions in grounded environments (Shrid-har et al., 2020; Stepputtis et al., 2020; Min et al., 2022b; Weir et al., 2022) and database commands (Kim et al., 2020).”
RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation (2024, unknown): “We consider the concrete task of language-conditioned bimanual manipulation with vision, which is fundamental in robotics and has great value in real-world scenarios such as household (Stepputtis et al., 2020; Brohan et al., 2022; Zhao et al., 2023).”
BridgeData V2: A Dataset for Robot Learning at Scale (2023, methodology): “As a baseline, we evaluate a standard languageconditioned behavior cloning method [55].”
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation (2021, methodology): “Other works have studied using offline data in the form of demonstrations [7] or human teleoperated trajectories (i.”
Code as Policies: Language Model Programs for Embodied Control (2022, unknown): “, model-based [33]–[35], imitation learning [8], [9], [36], [37], or reinforcement learning [38]–[42]).”
ChatGPT for Robotics: Design Principles and Model Abilities (2023, background): “More recent works explore neural networks to implicitly keep track of the complex mapping between language and actions, but such techniques often require vast amounts of labeled data for training [9], [10], [29], [30]”
Interactive Language: Talking to Robots in Real Time (2022, background): “prior work in broader AI research [1]–[6], [11] has sought a more convenient form of specification in the form of natural language conditioning (survey [27]), with some results on physical robots [7]–[9], [12].”