Asynchronous Methods for Deep Reinforcement Learning

By Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu. 2016

10,169 citations recorded by the source corpus.

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Pieter Abbeel
  5. Ruslan Salakhutdinov
  6. Quoc V. Le
  7. Jianfeng Gao
  8. Percy Liang

In Electrical and Electronic Engineering

  1. Sergey Levine
  2. Pieter Abbeel
  3. Shimon Whiteson
  4. Ofir Nachum
  5. Yoshua Bengio
  6. Chelsea Finn
  7. J. Urgen Schmidhuber
  8. Yuxi Li

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Shimon Whiteson
  5. Ofir Nachum
  6. Yuxi Li
  7. Shixiang Shane Gu
  8. Abhishek Gupta

Citations in the corpus