Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
By Felix Leibfried, Peter Vrancx. 2018
7 citations recorded by the source corpus.
This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmented objective leads to a richer training signal that provides feedback at every time step. Moreover, because learning an environment model shares a common structure with the RL problem, we hypothesize that the resulting objective improves both sample efficiency and performance. We empirically confirm our hypothesis on a range of 20 games from the Atari benchmark attaining superior results over vanilla DQN without model-based regularization.
Subfields: Reinforcement Learning, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Reinforcement Learning
- Sergey Levine
- Chelsea Finn
- Pieter Abbeel
- George Tucker
- Shixiang Shane Gu
- Ofir Nachum
- Aviral Kumar
- Mohammad Babaeizadeh
In Optimization & Training
- Chelsea Finn
- George Tucker
- Sergey Levine
- Roy H. Campbell
- Dumitru Erhan
- Piotr Milo's
- Ryan Sepassi
- Lukasz Kaiser
Citations in the corpus
- Model-Based Reinforcement Learning for Atari (2019, methodology): “Our method achieves around 330% of the Dyna-DQN score on Asterix, 120% on Q-Bert, 150% on Seaquest and 80% on Ms. Pac-Man.”