Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning

By Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine, Jonathan Tompson. 2018

111 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning via Fenchel-Rockafellar Duality (2020, background): “Many existing imitation learning works (Ho and Ermon, 2016; Fu et al., 2017; Kostrikov et al., 2018; Ke et al., 2019; Ghasemipour et al., 2019) aim to exactly optimize this f -divergence objective.”
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems (2019, methodology): “…imitation learning method, however standard GAIL does not work in the following settings: 1) POMDPs (Gangwani et al., 2019; ˙Zołna et al., 2019), 2) from pixels (Li et al., 2017; Reed et al., 2018), 3) off policy (Kostrikov et al., 2018) and 4) with variable initial conditions (Zolna et al., 2019).”
Imitation Learning via Off-Policy Distribution Matching (2019, methodology): “At the same time, one can verify that in this formulation the optimal π still matches π exp , unlike other proposals for incorporating a replay buffer distribution (Kostrikov et al., 2019).”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, background): “To encompass the episodic concept in inﬁnite MDPs, episodic RL tasks usually assume the existence of a set of absorbing states S 0 , which indicates the termination of episodic tasks [18], [19], and any action taken upon an absorbing state will only transit to itself with zero rewards.”
Primal Wasserstein Imitation Learning (2020, methodology): “…replacing the f -divergence by the Wasserstein distance through the Kantorovich-Rubinstein duality [Villani, 2008] leads to better training stability, which a number of methods in IL have leveraged [Li et al., 2017, Lacotte et al., 2019, Kostrikov et al., 2019, Xiao et al., 2019, Liu et al., 2020].”
Imitation Learning from Video by Leveraging Proprioception (2019, methodology): “Even more recently, there has been research on methods that seek to improve on GAIL by, e.g., increasing sample efﬁciency [ Kostrikov et al. , 2019; Sasaki et al. , 2019 ] and improving reward representation [ Fu et al. , 2018; Qureshi et al. , 2019].”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, unknown): “Over time, various enhancements have been proposed to GAIL, aiming to enhance its sample efﬁciency, scalability, and robustness [76], including modiﬁcations to the discriminator’s loss function [77] and transitioning from the on-policy to off-policy agents [78].”
Positive-Unlabeled Reward Learning (2019, methodology): “Using GAIL with an off-policy learning method strictly speaking requires importance sampling correction (Kostrikov et al., 2018), but we did not ﬁnd this to be necessary.”