Generative predecessor models for sample-efficient imitation learning
By Yannick Schroecker, Mel Vecerík, Jonathan Scholz. 2019
32 citations recorded by the source corpus.
We propose Generative Predecessor Models for Imitation Learning (GPRIL), a novel imitation learning algorithm that matches the state-action distribution to the distribution observed in expert demonstrations, using generative models to reason probabilistically about alternative histories of demonstrated states. We show that this approach allows an agent to learn robust policies using only a small number of expert demonstrations and self-supervised interactions with the environment. We derive this approach from first principles and compare it empirically to a state-of-the-art imitation learning method, showing that it outperforms or matches its performance on two simulated robot manipulation tasks and demonstrate significantly higher sample efficiency by applying the algorithm on a real robot.
Subfields: Artificial Intelligence, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Stefano Ermon
- Yoshua Bengio
- Chelsea Finn
- Ruslan Salakhutdinov
- Tengyu Ma
- Balaji Lakshminarayanan
In Control and Systems Engineering
- Sergey Levine
- Tengyu Ma
- Huazhe Xu
- Carlos Florensa
- Mariano Phielipp
- Pieter Abbeel
- Yuping Luo
- Yiming Ding
Citations in the corpus
- Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling (2019, background): “(2017); Schroecker et al. (2018) successfully combine generative models in the setting where a large amount of environment interaction without rewards are allowed.”
- Normalizing Flows for Probabilistic Modeling and Inference (2019, background): “Imitation learning Schroecker et al. (2019) use normalizing flows within the imitationlearning paradigm for control problems.”
- Goal-conditioned Imitation Learning (2019, methodology): “Originally, the algorithms used to optimize the policy are on-policy methods like Trust Region Policy Optimization [36], but recently there has been a wake of works leveraging the efficiency of off-policy algorithms without loss in stability [37, 38, 39, 40].”