Comparing Policy-Gradient Algorithms

By Richard S. Sutton, Satinder Singh, David A. McAllester. 2001

31 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

1. Sergey Levine
2. Shimon Whiteson
3. Pieter Abbeel
4. Nicolas Manfred Otto Heess
5. David Silver
6. Doina Precup
7. Ofir Nachum
8. Shixiang Shane Gu

In Optimization & Training

1. Shimon Whiteson
2. Doina Precup
3. Daan Wierstra
4. David Silver
5. Thomas Degris
6. Kamil Andrzej Ciosek
7. Martin Klissarov
8. Nicolas Manfred Otto Heess

Citations in the corpus

• Expected Policy Gradients for Reinforcement Learning (2018, methodology): “If π is parameterised by θ , then stochastic policy gradients (SPG) (Sutton et al., 2000; Peters and Schaal, 2006, 2008b) perform gradient ascent on ∇ J , the gradient of J with respect to θ (gradients without a subscript are always with respect to θ ).”
• Flexible Option Learning (2021, background): “Indeed, the update rule in Equation 7 can be seen as a variation on the all-action policy gradient [Sutton et al., 1983, Petit et al., 2019], which updates all actions applied to the distribution of options.”
• Deterministic Policy Gradient Algorithms (2014, methodology): “…ef-ficiently by temporal-difference learning (Bhatnagar et al., 2007; Degris et al., 2012b; Peters et al., 2005); indeed if both i) and ii) are satisfied then the overall algorithm is equivalent to not using a critic at all (Sutton et al., 2000), much like the REINFORCE algorithm (Williams, 1992).”