Task-Embedded Control Networks for Few-Shot Imitation Learning

By Stephen James, Michael Bloesch, Andrew J. Davison. 2018

150 citations recorded by the source corpus.

Much like humans, robots should have the ability to leverage knowledge from previously learned tasks in order to learn new tasks quickly in new and unfamiliar environments. Despite this, most robot learning approaches have focused on learning a single task, from scratch, with a limited notion of generalisation, and no way of leveraging the knowledge to learn other tasks more efficiently. One possible solution is meta-learning, but many of the related approaches are limited in their ability to scale to a large number of tasks and to learn further tasks without forgetting previously learned ones. With this in mind, we introduce Task-Embedded Control Networks, which employ ideas from metric learning in order to create a task embedding that can be used by a robot to learn new tasks from one or more demonstrations. In the area of visually-guided manipulation, we present simulation results in which we surpass the performance of a state-of-the-art method when using only visual information from each demonstration. Additionally, we demonstrate that our approach can also be used in conjunction with domain randomisation to train our few-shot learning ability in simulation and then deploy in the real world without any additional training. Once deployed, the robot can learn new tasks from a single real-world demonstration.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology, Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

RoboNet: Large-Scale Multi-Robot Learning (2019, background): “A number of works have studied learning representations and policies that transfer across domains, including transfer from simulation to the real world [1, 33, 34], transfer across different dynamics [35, 36, 37, 2], transfer across robot morphologies with invariant feature spaces [38] and modularit”
The MAGICAL Benchmark for Robust Imitation (2020, background): “Meta-IL [15, 25] and meta-IRL [38, 39] algorithms assume that a large body of demonstrations is available for some set of “train tasks”, but only a few demonstrations are available for “test tasks” that might be encountered in the future.”
RLBench: The Robot Learning Benchmark & Learning Environment (2019, background): “[6] there is a very narrow distribution of tasks, where the task of “placing a peach into a red bowl” would be considered a different task to “placing an apple in to a green bowl”.”
Watch, Try, Learn: Meta-Learning from Demonstrations and Reward (2019, methodology): “Unlike prior work in fewshot imitation learning [6, 9, 43, 17, 27], our method enables the agent to additionally improve upon trial-and-error experience.”
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables (2019, methodology): “Contextual methods have also been applied to imitation learning by conditioning the policy on a learned embedding of a demonstration and optimizing with behavior cloning (Duan et al., 2017; James et al., 2018).”
Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks (2018, background): “Domain randomization [51, 61, 25, 38, 3, 24] is another technique that is particularly popular in robotics, where an agent is trained on a wide range of variations of sensory inputs, with the intention that this forces the input processing layers of the network to extract semantically relevant featu”
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “Many of these works adapt to the new scenario by condi59 tioning on a robot demonstration [1, 2], a video of a human [3, 4], a language instruction [23, 24], 60 or a goal image [21].”
Parrot: Data-Driven Behavioral Priors for Reinforcement Learning (2020, background): “…et al., 2019; Zintgraf et al., 2020; Fakoor et al., 2020) and meta-imitation methods (Duan et al., 2017; Finn et al., 2017c; Huang et al., 2018; James et al., 2018; Paine et al., 2018; Yu et al., 2018; Huang et al., 2019; Zhou et al., 2020) also seek to speed up learning for new tasks by…”
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale (2021, unknown): “Meta-learning has been both combined with imitation learning [18, 25, 83, 34, 8, 52, 84] and reinforcement learning through context space learning [79, 17, 47, 59, 90] and gradient-based optimization [24, 61, 33, 28, 46].”
PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale (2022, background): “One family of approaches for multitask robot learning focuses on supervised learning of multitask control policies to maximize few-shot performance via meta-learning [23, 24, 25] or direct zero-shot performance via behavioral cloning [26, 27, 28, 29].”
Multi-Task Learning with Deep Neural Networks: A Survey (2020, methodology): “(James et al., 2018) uses metric learning to construct a task embedding for imitation learning of various robotic manipulation tasks.”
Provable Representation Learning for Imitation Learning via Bi-level Optimization (2020, methodology): “A line of work uses meta-learning for imitation learning (Duan et al., 2017; Finn et al., 2017b; James et al., 2018).”
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (2023, methodology): “Hence we refer to our model as RT-2 (Robotics Transformer 2).”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “, 2017), new objects (Finn et al., 2017b; James et al., 2018), and new terrains or environment conditions (Clavera et al.”
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation (2024, methodology): “Prior works also focus on multi-task or few-shot imitation learning, [25, 27, 30, 34, 46, 50, 88, 102], language-conditioned imitation learning [12, 47, 82, 83], imitation from play data [21, 57, 74, 89], using human videos [16, 24, 29, 60, 69, 80, 84, 96], and using task-specific structures [49,…”