On-Policy vs. Off-Policy Updates for Deep Reinforcement Learning

By M. Stone. 2016

30 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

1. Sergey Levine
2. Aviral Kumar
3. Pieter Abbeel
4. Ofir Nachum
5. Justin Fu
6. Chelsea Finn
7. George Tucker
8. Shixiang Shane Gu

In Optimization & Training

1. Justin Fu
2. Aviral Kumar
3. Brendan O'Donoghue
4. Alex Smola
5. Pratik Chaudhari
6. Rasool Fakoor
7. R´emi Munos
8. Sergey Levine

Citations in the corpus

• PGQ: Combining policy gradient and Q-learning (2016, methodology): “We conclude with some numerical examples that provide empirical evidence of improved data efficiency and stability of PGQ.”
• Diagnosing Bottlenecks in Deep Q-learning Algorithms (2019, methodology): “In deep RL, several methods have been developed to prevent instabilities in Q-Learning, such as prioritized replay (Schaul et al., 2015), and mixing replay buffer with on-policy data (Hausknecht & Stone, 2016; Zhang & Sutton, 2017) have been found to be beneficial.”
• P3O: Policy-on Policy-off Policy Optimization (2019, background): “Let us note there are a number of works that learn action-value functions using off-policy data, e.g.,Wang et al. (2013); Hausknecht and Stone (2016); Lehnert and Precup (2015) that achieve varying degrees of success on reinforcement learning benchmarks.”