Generative Adversarial Self-Imitation Learning

By Yijie Guo, Junhyuk Oh, Satinder Singh, Honglak Lee. 2018

61 citations recorded by the source corpus.

This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproducing past good trajectories, which can potentially make long-term credit assignment easier when rewards are sparse and delayed. GASIL can be easily combined with any policy gradient objective by using GASIL as a learned shaped reward function. Our experimental results show that GASIL improves the performance of proximal policy optimization on 2D Point Mass and MuJoCo environments with delayed reward and stochastic dynamics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jian Peng
2. Qiang Liu
3. Yang Liu
4. Sergey Levine
5. Xi Chen
6. Yunan Luo
7. Yuanyi Zhong
8. Pieter Abbeel

In Computer Vision and Pattern Recognition

1. Jian Peng
2. Qiang Liu
3. Yang Liu
4. Xi Chen
5. Sergey Levine
6. Pieter Abbeel
7. Luc Van Gool
8. Dacheng Tao

Citations in the corpus

• Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning (2019, background): “Unfortunately, when the reward signal becomes delayed or even episodic, most existing deep reinforcement learning algorithms may get stuck during the training process and often suffer from inferior performance and inefficient sample complexity [8, 9].”