Skill Induction and Planning with Latent Language

By Pratyusha Sharma, Antonio Torralba, Jacob Andreas. 2021

129 citations recorded by the source corpus.

We present a framework for learning hierarchical policies from demonstrations, using sparse natural language annotations to guide the discovery of reusable skills for autonomous decision-making. We formulate a generative model of action sequences in which goals generate sequences of high-level subtask descriptions, and these descriptions generate sequences of low-level actions. We describe how to train this model using primarily unannotated demonstrations by parsing demonstrations into sequences of named high-level sub-tasks, using only a small number of seed annotations to ground language in action. In trained models, natural language commands index a combinatorial library of skills; agents can use these skills to plan by generating high-level instruction sequences tailored to novel goals. We evaluate this approach in the ALFRED household simulation environment, providing natural language annotations for only 10% of demonstrations. It achieves performance comparable state-of-the-art models on ALFRED success rate, outperforming several recent methods with access to ground-truth plans during training and evaluation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In LLM Agents & Tool Use

Citations in the corpus

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, background): “A large number of prior works have learned language-conditioned behavior via imitation learning [51, 22, 20, 13, 26, 37] or reinforcement learning [52, 53, 49, 54, 55, 56, 21, 41].”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “Various prior works have explored using language as a space for planning [51, 52, 53, 20, 54, 21].”
Pre-Trained Language Models for Interactive Decision-Making (2022, background): “Semi-Supervised Skill Learning with Latent Language (SL) 3 [38] shows that LMs can serve as an effective backbone for hierarchical policies that express plans as natural language strings [2, 4].”
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2022, background): “Recent works also use language as hierarchical abstractions to plan actions using imitation learning (Sharma et al., 2021) and to guide exploration in reinforcement learning (Mirchandani et al.”
PaLM-E: An Embodied Multimodal Language Model (2023, background): “(SL)(3) (Sharma et al., 2021) tackles the more challenging task of simultaneously finetuning two LLMs: a planning network, which produces high-level instructions, and a low-level policy network, which selects actions.”
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, methodology): “In reinforcement learning settings, generating natural language has been used as an intermediate planning step (Sharma et al., 2021; Hu et al., 2019), and in particular in goal-oriented dialogue (Yarats and Lewis, 2018) and open-domain QA (Adolphs et al., 2021) as well.”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “The comparative summary of the embodied robots, simulators, datasets, and approaches provides a clear picture of the recent development in embodied AI, which greatly beneﬁts the future research along this emerging and promising research direction.”
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (2023, background): “Recently, researchers have used learning-based methods to learn planning policies directly [22] or to learn search heuristics to accelerate planning [32, 31, 40, 7, 29].”
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models (2023, unknown): “…agents [19–22] and of robotics [23, 6, 24, 25, 5, 7, 26], where language can be used as a tool for compositional goal specification [5, 27–33], semantic anchor for training multi-modal representation [12, 34, 35], or as an intermediate substrate for planning and reasoning [36– 38, 9, 10, 39, 40].”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “(SL) 3 [155] is a learning algorithm that alternates between three steps: segmentation, labeling, and parameter update.”
Building Cooperative Embodied Agents Modularly with Large Language Models (2023, background): “…face challenges when solving complex reasoning problems (Bubeck et al., 2023), a substantial body of work demonstrates their capacity to make plans (Sharma et al., 2021; Raman et al., 2022; Pallagani et al., 2022; Gramopadhye & Szafir, 2022; Yuan et al., 2023; Li et al., 2022; Wang et al., 2023d),…”
Vygotskian Autotelic Artificial Intelligence: Language and Culture Internalization for Human-Like AI (2022, background): “88,100,104–108 Among them, Wong et al. use plan generation as an auxiliary task to train a drawing policy.”