An Optimistic Perspective on Offline Deep Reinforcement Learning

By Rishabh Agarwal, Dale Schuurmans, Dale Schuurmans, Mohammad Norouzi. 2020

22 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Decision Transformer: Reinforcement Learning via Sequence Modeling (2021, methodology): “We compare to CQL [14], REM [13], and QR-DQN [22] on four Atari tasks (Breakout, Qbert, Pong, and Seaquest) that are evaluated in Agarwal et al.”
Accelerating Online Reinforcement Learning with Offline Datasets (2020, positive): “Offline RL methods aim to address this issue, often by constraining the actor updates to avoid excessive deviation from the data distribution [24, 43, 15, 16, 14, 2, 23, 10, 8, 28, 40, 25, 51].”
Continuous Doubly Constrained Batch Reinforcement Learning (2021, methodology): “[25, 64] consider model-based approaches for batch RL, and [1] find ensembles partly address some of the issues that arise in batch”
RL Unplugged: Benchmarks for Offline Reinforcement Learning (2020, methodology): “It is a popular benchmark to measure the progress of online RL methods, and Atari has recently also become a standard benchmark for offline RL methods (Agarwal et al., 2020; Fujimoto et al., 2019) as well.”
QPLEX: Duplex Dueling Multi-Agent Q-Learning (2020, background): “Agarwal et al. (2020) presents an optimistic perspective of offline Q-learning that DQN and its variants can achieve superior performance in Atari 2600 games (Bellemare et al.”
What are the Statistical Limits of Offline RL with Linear Function Approximation? (2020, methodology): “…recently, there are an increasing number of both model-free [Laroche et al., 2019, Fujimoto et al., 2019, Jaques et al., 2020, Kumar et al., 2019, Agarwal et al., 2020] and model-based [Ross and Bagnell, 2012, Kidambi et al., 2020] offline RL methods, with steady improvements in performance…”
Autoregressive Dynamics Models for Offline Policy Evaluation and Optimization (2021, background): “Even though offline RL offers the promise of leveraging existing logged datasets, current offline RL algorithms (Fujimoto et al., 2019; Agarwal et al., 2020; Kumar et al., 2019) are typically evaluated using online interaction, which limits their applicability in the real world.”
PLAS: Latent Action Space for Offline Reinforcement Learning (2020, background): “Recent work proposes different approaches in this direction [9, 10, 11, 12, 13].”
Provably Good Batch Reinforcement Learning Without Great Exploration (2020, background): “Another recent work [2] highlights that sufficiently large and diverse datasets can lead to good batch RL results but we focus on the other side of the coin: a robust update rule even if we do not have a good dataset.”