Watch, Try, Learn: Meta-Learning from Demonstrations and Reward

By Allan Zhou, Eric Jang, Daniel Kappler, Alexander Herzog, Mohi Khansari, Paul Wohlhart, Yunfei Bai, Mrinal Kalakrishnan, Sergey Levine, Chelsea Finn. 2019

54 citations recorded by the source corpus.

Imitation learning allows agents to learn complex behaviors from demonstrations. However, learning a complex vision-based task may require an impractical number of demonstrations. Meta-imitation learning is a promising approach towards enabling agents to learn a new task from one or a few demonstrations by leveraging experience from learning similar tasks. In the presence of task ambiguity or unobserved dynamics, demonstrations alone may not provide enough information; an agent must also try the task to successfully infer a policy. In this work, we propose a method that can learn to learn from both demonstrations and trial-and-error experience with sparse reward feedback. In comparison to meta-imitation, this approach enables the agent to effectively and efficiently improve itself autonomously beyond the demonstration data. In comparison to meta-reinforcement learning, we can scale to substantially broader distributions of tasks, as the demonstration reduces the burden of exploration. Our experiments show that our method significantly outperforms prior approaches on a set of challenging, vision-based control tasks.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning (2022, background): “Multiple prior imitation learning works have achieved different forms of generalization, including one-shot generalization to novel objects [1, 2, 3, 4, 18], to novel object configurations [19], and to novel goal configurations [6, 7, 20], as well as zero-shot generalization to new objects [5], scen”
Transformers for One-Shot Visual Imitation (2020, methodology): “Speciﬁcally, 4 baseline architectures are considered: 2 of them adopt the small convolutional network used in prior work [10, 57] either with or without an additional LSTM [52] on top, and the other 2 use ResNet features [43] (again with or without LSTM).”
One-Shot Learning from a Demonstration with Hierarchical Latent Language (2022, unknown): “Speciﬁcally, our experimental setup falls under the zero-and few-shot imitation learning category (Duan et al., 2017; Yu et al., 2018; Pan et al., 2020; Zhou et al., 2020), where our approach receives a single demonstration to solve novel tasks.”
Parrot: Data-Driven Behavioral Priors for Reinforcement Learning (2020, background): “, 2020) and meta-imitation methods (Duan et al., 2017; Finn et al., 2017c; Huang et al., 2018; James et al., 2018; Paine et al., 2018; Yu et al., 2018; Huang et al., 2019; Zhou et al., 2020) also seek to speed up learning for new tasks by leveraging experience from previously seen tasks.”