Provable Representation Learning for Imitation Learning via Bi-level Optimization
By Sanjeev Arora, Simon Shaolei Du, Sham M. Kakade, Yuping Luo, Nikunj Saunshi. 2020
65 citations recorded by the source corpus.
A common strategy in modern learning systems is to learn a representation that is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We formulate representation learning as a bi-level optimization problem where the "outer" optimization tries to learn the joint representation and the "inner" optimization encodes the imitation learning setup and tries to learn task-specific parameters. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone. Theoretically, we show using our framework that representation learning can provide sample complexity benefits for imitation learning in both settings. We also provide proof-of-concept experiments to verify our theory.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mengjiao Yang
- Ofir Nachum
- Sergey Levine
- Shixiang Shane Gu
- George Tucker
- Pieter Abbeel
- Aviral Kumar
- Ilya Kostrikov
Citations in the corpus
- Provable Representation Learning for Imitation with Contrastive Fourier Features (2021, background): “One previous line of research in this vein is given by [9], which considers”