Investigating Recurrence and Eligibility Traces in Deep Q-Networks

By Jean Harb, Doina Precup. 2017

22 citations recorded by the source corpus.

Eligibility traces in reinforcement learning are used as a bias-variance trade-off and can often speed up training time by propagating knowledge back over time-steps in a single update. We investigate the use of eligibility traces in combination with recurrent networks in the Atari domain. We illustrate the benefits of both recurrent nets and eligibility traces in some Atari games, and highlight also the importance of the optimization used in the training.

Subfields: Artificial Intelligence, Reinforcement Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Wojciech M. Czarnecki
3. Karl Tuyls
4. Thore Graepel
5. Joel Z. Leibo
6. Marc Lanctot
7. Max Jaderberg
8. Shimon Whiteson

In Reinforcement Learning

1. Karl Tuyls
2. Shimon Whiteson
3. Joel Z. Leibo
4. Thore Graepel
5. Marc Lanctot
6. Wojciech M. Czarnecki
7. Sergey Levine
8. Vinícius Flores Zambaldi

In Model Architectures

1. Wojciech M. Czarnecki
2. Thore Graepel
3. Joel Z. Leibo
4. Marc Lanctot
5. Audrunas Gruslys
6. Karl Tuyls
7. Max Jaderberg
8. Vinícius Flores Zambaldi

Citations in the corpus

• Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, methodology): “To speed up learning we add the dueling architecture of Wang et al. (2016) that represent Q using a value and an advantage function, including multi-step updates with a forward view eligibility trace (e.g. Harb and Precup, 2016) over a certain number of steps.”