A Closer Look at Deep Policy Gradients

By Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, Aleksander Madry. 2018

110 citations recorded by the source corpus.

Subfields: Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

1. Pratik Chaudhari
2. Rasool Fakoor
3. Alex Smola
4. Yu Wang
5. Alexandre M. Bayen
6. Chao Yu
7. Eugene Vinitsky
8. Stefano Soatto

Citations in the corpus

• The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (2021, background): “Furthermore, similar to the suggestions by [17], we find that using more data to estimate gradients typically leads to improved practical performance.”
• P3O: Policy-on Policy-off Policy Optimization (2019, background): “Further, they tend to have high variance gradient estimates which necessitates a large number of online samples and highly distributed training (Ilyas et al., 2018; Mnih et al., 2016).”
• Meta-Q-Learning (2019, methodology): “Policy-gradient-based algorithms versus off-policy methods: Policy-gradient-based methods have high sample complexity (Ilyas et al., 2018).”
• The Surprising Effectiveness of MAPPO in Cooperative, Multi-Agent Games (2021, background): “Furthermore, similar to the suggestions by (Ilyas et al., 2020), we always find that using more data to estimate gradients leads to improved practical performance; thus, we do not split training data into mini-batches by default.”