Dueling Network Architectures for Deep Reinforcement Learning

By Ziyun Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot, Nando de Freitas. 2015

4,476 citations recorded by the source corpus.

In recent years there have been many successes of using deep representations in reinforcement learning. Still, many of these applications use conventional architectures, such as convolutional networks, LSTMs, or auto-encoders. In this paper, we present a new neural network architecture for model-free reinforcement learning. Our dueling network represents two separate estimators: one for the state value function and one for the state-dependent action advantage function. The main benefit of this factoring is to generalize learning across actions without imposing any change to the underlying reinforcement learning algorithm. Our results show that this architecture leads to better policy evaluation in the presence of many similar-valued actions. Moreover, the dueling architecture enables our RL agent to outperform the state-of-the-art on the Atari 2600 domain.

Subfields: Artificial Intelligence, Automotive Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Pieter Abbeel
  4. Mohit Bansal
  5. Ruslan Salakhutdinov
  6. Quoc V. Le
  7. Jianfeng Gao
  8. Percy Liang

In Automotive Engineering

  1. Sergey Levine
  2. Pieter Abbeel
  3. Lei Tai
  4. R´emi Munos
  5. Chelsea Finn
  6. Ofir Nachum
  7. Marc G. Bellemare
  8. David Silver

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Shimon Whiteson
  4. Yuxi Li
  5. Chelsea Finn
  6. Ofir Nachum
  7. Marc G. Bellemare
  8. Shixiang Shane Gu

Citations in the corpus