Do recent advancements in model-based deep reinforcement learning really improve data efficiency?

By Kacper Kielak. 2019

50 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Evaluation & Benchmarks

Citations in the corpus

Model-Based Reinforcement Learning for Atari (2019, background): “Since the publication of the ﬁrst preprint of this work, it has been shown in van Hasselt et al. (2019); Kielak (2020) that Rainbow can be tuned to have better results in low data regime.”
CURL: Contrastive Unsupervised Representations for Reinforcement Learning (2020, methodology): “been common practice when investigating data-efficiency on Atari (Kaiser et al., 2019; van Hasselt et al., 2019; Kielak, 2020), the DMControl benchmark was set at 500k environment steps because state-based RL approaches asymptotic performance on many environments at this point, and 100k steps to mea”
Deep Reinforcement Learning at the Edge of the Statistical Precipice (2021, methodology): “As a concrete example, our analysis suggests that DER may in fact be better than OTR, unlike what the reported point estimates suggest.”
Randomized Ensembled Double Q-Learning: Learning Fast Without a Model (2021, background): “Although in discrete action domains it has been found that ﬁne-tuning DQN variants, including the UTD, can boost performance, little experimental or theoretical analysis is given to explain how this improvement is obtained (Kielak, 2020; van Hasselt et al., 2019).”
Mastering Atari Games with Limited Data (2021, methodology): “However, Kielak [19] and van Hasselt et al. [39] argue that this is not necessary to achieve strong results with model-based methods, and they show that when tuned appropriately, Rainbow [16] can achieve comparable results.”
Data-Efficient Reinforcement Learning with Self-Predictive Representations (2020, result): “We ﬁnd that our controlled Rainbow implementation without augmentation is slightly stronger than Data-Efﬁcient Rainbow but comparable to Overtrained Rainbow (Kielak, 2020), while with augmentation enabled our results are somewhat stronger than DrQ. 2 None of these methods, however, are close to the ”