One-Shot Visual Imitation Learning via Meta-Learning

By Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, Sergey Levine. 2017

620 citations recorded by the source corpus.

In order for a robot to be a generalist that can perform a wide range of jobs, it must be able to acquire a wide variety of skills quickly and efficiently in complex unstructured environments. High-capacity models such as deep neural networks can enable a robot to represent complex skills, but learning each skill from scratch then becomes infeasible. In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration. Unlike prior methods for one-shot imitation, our method can scale to raw pixel inputs and requires data from significantly fewer prior tasks for effective learning of new skills. Our experiments on both simulated and real robot platforms demonstrate the ability to learn new tasks, end-to-end, from a single visual demonstration.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (2017, background): “However, as shown in Finn et al. (2017b) and similar to the setup in Neu & Szepesv´ari (2012), we actually have considerable ﬂexibility here to make the inner and outer objectives different.”
Meta-Gradient Reinforcement Learning (2018, neutral): “Model-Agnostic Meta-Learning (MAML) [Finn et al., 2017a, Finn and Levine, 2018, Finn et al., 2017b, Grant et al., 2018, Al-Shedivat et al., 2018] learns a good initialisation of the model that can adapt quickly to other tasks within a few gradient update steps.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “• Meta-learning : Finn et al. (2017a) and Nichol and Schulman (2018) propose methods that can effectively lead the policy search to ﬁnd parameters that can be adapted to give good performance on a new task with only a small number of training examples of the new task.”
Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm (2017, methodology): “In contrast to the aforementioned approaches, more recent work has proposed methods that include the structure of optimization problems into the meta-learner (Ravi & Larochelle, 2017; Finn et al., 2017a; Husken & Goerick, 2000).”
Sample Efficient Adaptive Text-to-Speech (2018, methodology): “This method has shown promise in robotics (Finn et al., 2017b; Yu et al., 2018).”
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, background): “Imitation learning has been applied to multiple domains such as playing table tennis [22], Go [37] and video games [33], and driving autonomously [4, 31] In robotics, the idea of robots learning from human demonstrations have been extensively explored [36, 17, 23, 11, 14, 3, 5].”
RoboNet: Large-Scale Multi-Robot Learning (2019, background): “A number of works have studied learning representations and policies that transfer across domains, including transfer from simulation to the real world [1, 33, 34], transfer across different dynamics [35, 36, 37, 2], transfer across robot morphologies with invariant feature spaces [38] and modularit”
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Recent works [7, 19, 42] use deep neural networks to map directly from image observations to action, and have proven effective for learning visualmotor skills for complex and diverse manipulation tasks.”
RLBench: The Robot Learning Benchmark & Learning Environment (2019, background): “2974707 there is now a large body of work looking at increasing the capabilities of robotic agents through the use of reinforcement learning [3], [4], meta-learning [5]–[7], multi-task learning [8],”
Self-Supervised Correspondence in Visuomotor Policy Learning (2019, background): “The model is close to [1]–[3] with the key architectural traits of having a few convolutional layers followed by a channel-wise spatial expectation operation, which has been widely used [10], [11], [25], [29], [36]–[38].”
Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation (2018, neutral): “Imitation learning using data from multiple tasks has also shown promise [39].”
Reinforcement and Imitation Learning for Diverse Visuomotor Skills (2018, methodology): “Recently, methods for third person imitation have been proposed (e.g. Sermanet et al. [44], Liu et al. [27], Finn et al. [7]).”
Taskonomy: Disentangling Task Transfer Learning (2018, background): “Meta-learning generally seeks performing the learning at a level higher than where conventional learning occurs, e.g. as employed in reinforcement learning [21, 31, 28], optimization [2, 82, 48], or certain architectural mechanisms [27, 30, 87, 65].”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “To deal with high-dimensional data such as raw images, a deep learning approach is employed for modeling a forward dynamics in the most recent studies [Oh et al., 2015, Finn et al., 2017a, Baram et al., 2017, Nair et al., 2017].”
Learning Latent Plans from Play (2019, methodology): “[15]), our method does not require any expensive task-specific demonstrations for training or a predefined task distribution, only non-specific play data.”
Playing hard exploration games by watching YouTube (2018, neutral): “There have also been many previous studies that consider supervised [4] and few-shot methods [13, 15] for imitation learning.”
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning (2018, methodology): “…improvement in success by using a temporal adaptation objective, indicating the importance of integrating temporal information when learning from raw video. simulated pushing no domain shift LSTM [10] 34.23% contextual 56.98% MIL, linear loss [15] 66.44% MIL, temporal loss (ours) 80.63% VII.”