Policy Optimization with Demonstrations

By Bingyi Kang, Zequn Jie, Jiashi Feng. 2018

174 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Measuring the divergence between the current policy and the expert policy for optimization is proposed in [68].”
A Minimalist Approach to Offline Reinforcement Learning (2021, methodology): “…using BC with RL, imitation learning has been combined with RL in a variety of manners, such as mixed with adversarial methods [Zhu et al., 2018, Kang et al., 2018], used for pre-training [Pfeiffer et al., 2018], modifying the replay buffer [Večerík et al., 2017, Gulcehre et al., 2020] or…”
TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations (2021, neutral): “Training agents from pre-collected dataset has long been studied (Ho and Ermon, 2016; Kang, Jie, and Feng, 2018; Oh et al., 2018; Peng et al., 2019; Fu et al., 2020; Qin et al., 2021).”
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, background): “There has been a growing interest in combining extrinsic sparse reward signals with imitation learning for guided exploration (Zhu et al., 2018; Kang et al., 2018; Le et al., 2018; Vecerı́k et al., 2017).”
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems (2019, background): “Many other works have shown that demonstrations can accelerate learning and address hard-exploration tasks (e.g. see Hester et al., 2018; Kim et al., 2013; Nair et al., 2018; Kang et al., 2018).”
Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search (2020, background): “…to reshape the reward distributions to steer the agent’s action selection and navigation towards the expected trajectories; [59, 84, 39, 31, 49, 38, 74] make use of external demonstrations from different sources such as human experts or near-optimal policy to achieve more efﬁcient exploration.”
Exploring applications of deep reinforcement learning for real-world autonomous driving systems (2019, background): “Measuring the divergence between the current policy and the expert for policy optimization is proposed in (Kang et al., 2018).”
IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks (2019, methodology): “In addition, we also evaluate the demonstration-guided RL approach, GAIL+PPO [39], which learns from the weighted sum of GAIL and task rewards.”
Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning (2018, background): “oses a signiﬁcant challenge in reinforcement learning. There has been a growing interest in combining extrinsic sparse reward signals with imitation learning for guided exploration (Zhu et al., 2018; Kang et al., 2018; Le et al., 2018; Vecer´ık et al., 2017). As in this work, off policy learning fro”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, methodology): “For example, [97] extended GAIL with an algorithm called Policy Optimization from Demonstrations (POfD) , which combines the discriminator reward with the environment reward: Both GAIL and POfD are under an on-policy RL framework.”
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos (2021, methodology): “To equip GAIL with reward function, we adopt the approach proposed by [34], which combines the discriminator feedback and reward signal via reward shaping.”