Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

By Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese. 2017

212 citations recorded by the source corpus.

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well towards unseen tasks with increasing lengths, variable topologies, and changing objectives.stanfordvl.github.io/ntp/.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “Machine learning has recently been applied to enable abstract task specification, allow general primitives, or relax constraints [67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78].”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “Traditionally, this problem has been solved through optimization [28, 29] or symbolic reasoning [24, 26], but more recently machine learning has been applied to aspects of the problem via learned representations, learned task-primitives, and more [30, 31, 32, 33, 34, 35, 36, 37, 38].”
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, methodology): “The task can be represented as a full video [41], a sequence of image keyframes [16], or a trajectory in state space [10], while the training objective can”
Modular meta-learning (2018, methodology): “Johnson et al. [13] later showed that the structure controller could be trained with RL; others applied similar frameworks to get more interpretability [14] or to generalize across hierarchical robotic tasks with neural programs[15].”
Automatically Composing Representation Transformations as a Means for Generalization (2018, background): “The ﬁrst as-Automatically sumes pre-speciﬁed transformations and learns the structure (from dense supervision on execution traces to sparse-rewards) [13, 14, 18, 22, 25, 28, 65, 87, 90].”
Lifelong Learning of Compositional Structures (2020, background): “The majority of such methods either learn the structure for piecing together a given set of components [30, 4, 5] or learn the set of components given a known structure for how to compose them [3].”
Prospection: Interpretable plans from language by predicting the future (2019, background): “Neural approaches and scaling robotic learning within simulation have become common as they allow for end-to-end training and can easily acquire more data (often from multiple domains) than otherwise possible on a physical device [2].”
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation (2019, background): “…to add compositional structure to policies, either from demonstrations (Krishnan et al., 2017; Fox et al., 2018), with manually-speciﬁed primitives (Xu et al., 2017; Huang et al., 2018), learned temporal abstractions (Neitz et al., 2018), or through model-free reinforcement learning (Sutton et…”
Learning Compositional Neural Programs with Recursive Tree Search and Planning (2019, unknown): “Neural Task Programming [Xu et al., 2018] adapted NPI to the setting of robotics in order to learn manipulation behaviors from visual demonstrations and annotations of the program hierarchy.”
Physically Embedded Planning Problems: New Challenges for Reinforcement Learning (2020, background): “These include a focus on dexterity (e.g., Akkaya et al., 2019; OpenAI et al., 2018), but also long-horizon, multi-step control (e.g., Duan et al., 2017; Gupta et al., 2019; Xu et al., 2017).”
Visual Robot Task Planning (2018, methodology): “One-shot deep imitation learning can produce general-purpose models for various tasks, but does not provide useful introspection on how to sequence different steps in an operation [2].”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “Other recent approaches have relied on observations only, learning to segment videos of multi-step tasks into composable primitives [127, 433, 165, 424].”
Modeling Long-horizon Tasks as Sequential Interaction Landscapes (2020, background): “More recently, Xu et al. [59] propose to hierarchically decompose tasks into sub-task speciﬁcations to then learn task-conditioned policies to interact with objects in an environment.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “A separate approach specifically for robot programming tasks uses the application programming interfaces (APIs) of primitive actions to guide the learning (Xu et al., 2018).”
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks (2019, background): “Domain Knowledge : Instruction manuals [45, 1], programs [26, 46], and video demonstrations [21] can be used as domain knowledge to help solving complex tasks.”
Neural Production Systems (2021, background): “Neural networks have been studied as a way to address the problems of learning procedural behavior and program induction (Graves et al., 2014; Reed & De Freitas, 2015; Neelakantan et al., 2015; Cai et al., 2017; Xu et al., 2018; Trask et al., 2018; Bunel et al., 2018; Li et al., 2020).”
Learning Robot Skills with Temporal Variational Inference (2020, methodology): “But even using demonstrations, approaches such as Xu et al. (2018); Huang et al. (2019) require heavy supervision such as segmentation of demonstration data.”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, methodology): “When available, demonstration can be an effective method for guiding exploration [27], [28], [29].”
Neurosymbolic Programming (2021, background): “An emerging research direction is to automatically learn or discover the structure of the symbolic knowledge (e.g., Xu et al. (2018)), which can be viewed as an instance of neurosymbolic programming.”