Imitation Learning via Kernel Mean Embedding
By Kee-Eung Kim, Hyun Soo Park. 2018
44 citations recorded by the source corpus.
Imitation learning refers to the problem where an agent learns a policy that mimics the demonstration provided by the expert, without any information on the cost function of the environment. Classical approaches to imitation learning usually rely on a restrictive class of cost functions that best explains the expert's demonstration, exemplified by linear functions of pre-defined features on states and actions. We show that the kernelization of a classical algorithm naturally reduces the imitation learning to a distribution learning problem, where the imitation policy tries to match the state-action visitation distribution of the expert. Closely related to our approach is the recent work on leveraging generative adversarial networks (GANs) for imitation learning, but our reduction to distribution learning is much simpler, robust to scarce expert demonstration, and sample efficient. We demonstrate the effectiveness of our approach on a wide range of high-dimensional control tasks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Olivier Pietquin
- Matthieu Geist
- L'eonard Hussenot
- Robert Dadashi
- Ilya Kostrikov
- Jonathan Tompson
- Pieter Abbeel
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Andrew Zisserman
- Sergey Levine
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Abhinav Kumar Gupta
In Statistical and Nonlinear Physics
- Ilya Kostrikov
- Minkai Xu
- Tairan He
- Weinan Zhang
- Minghuan Liu
- Sergey Levine
- Debidatta Dwibedi
- Jonathan Tompson
Citations in the corpus
- Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, methodology): “…we present examples of bias present in reward functions in different AIL algorithms: • In the GAIL framework, and follow-up methods, such as GMMIL (Kim & Park, 2018) and AIRL, zero reward is implicitly assigned for absorbing states, while some reward function, r ( s, a ) , assigns rewards for…”
- Energy-Based Imitation Learning (2020, methodology): “And we are unsuccessful with GMMIL and RED on Swimmer even after extensive tuning.”
- A Bayesian Approach to Generative Adversarial Imitation Learning (2018, background): “Among a number of improved variants of GAIL, we remark that generative moment matching imitation learning (GMMIL) [Kim and Park, 2018], which uses kernel mean embedding to improve the discriminator training just as in generative moment matching networks (GMMNs) [Li et al.”
- Offline Reinforcement Learning with Pseudometric Learning (2021, background): “Although pseudometrics in MDPs have proven to be effective in some applications (Melo & Lopes, 2010; Kim & Park, 2018; Dadashi et al., 2021), they are usually hand-crafted or learned with ad-hoc strategies.”
- Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation (2019, background): “Two broad categories of approaches to this settings are behavioral cloning (BC) Pomerleau (1991), which directly learns a policy mapping from states to actions with supervised learning from expert trajectories; and inverse reinforcement learning (IRL) Ng & Russell (2000); Abbeel & Ng (2004), which l”