Double Q-learning

By Hado van Hasselt. 2010

1,805 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Continuous control with deep reinforcement learning (2015, background): “On both low-dimensional and from pixels, some replicas were able to learn reasonable policies that are able to complete a circuit around the track though other replicas failed to learn a sensible policy.”
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018, methodology): “Our algorithm also makes use of two Q-functions to mitigate positive bias in the policy improvement step that is known to degrade performance of value based methods (Hasselt, 2010; Fujimoto et al., 2018).”
Addressing Function Approximation Error in Actor-Critic Methods (2018, methodology): “We additionally compare the effectiveness of the actor-critic variants of Double Q-learning (Van Hasselt, 2010) and Double DQN (Van Hasselt et al., 2016), denoted DQ-AC and DDQN-AC respectively, in Table 2.”
Taming the Noise in Reinforcement Learning via Soft Updates (2015, background): “The examples in Section 6 demonstrate the signiﬁcant improvement thus obtained.”
When to use parametric models in reinforcement learning? (2019, methodology): “…was to compare SimPLe to Rainbow DQN [Hessel et al., 2018a], which combines the DQN algorithm [Mnih et al., 2013, 2015] with double Q-learning [van Hasselt, 2010, van Hasselt et al., 2016], dueling network architectures [Wang et al., 2016], prioritised experience replay [Schaul et al., 2016],…”
Distributed Prioritized Experience Replay (2018, methodology): “More specifically, we used double Q-learning (van Hasselt, 2010; van Hasselt et al., 2016) with multi-step bootstrap targets (cf. Sutton, 1988; Sutton & Barto, 1998; 2017; Mnih et al., 2016) as the learning algorithm, and a dueling network architecture (Wang et al., 2016) as the function…”
Rainbow: Combining Improvements in Deep Reinforcement Learning (2017, methodology): “Double DQN (DDQN; van Hasselt, Guez, and Silver 2016) addresses an overestimation bias of Q-learning (van Hasselt 2010), by decoupling selection and evaluation of the bootstrap action.”
Prioritized Experience Replay (2015, methodology): “…experience replay, namely the version of the DQN algorithm from the Nature paper (Mnih et al., 2015), and its recent extension Double DQN (van Hasselt et al., 2016) that substantially improved the state-of-the-art by reducing the over-estimation bias with Double Q-learning (van Hasselt, 2010).”
Deep Reinforcement Learning with Double Q-Learning (2015, methodology): “Later van Hasselt (2010) argued that noise in the environment can lead to overestimations even when using tabular representation, and proposed Double Q-learning as a solution.”
Dueling Network Architectures for Deep Reinforcement Learning (2015, background): “This can therefore lead to overopti-mistic value estimates (van Hasselt, 2010).”
RUDDER: Return Decomposition for Delayed Rewards (2018, background): “DQN was extended to double DQN (DDQN) [134, 135] which helps exploration as the overestimation bias is reduced.”
Meta-Gradient Reinforcement Learning (2018, background): “…on Atari games, namely Rainbow [Hessel et al., 2018], which combines DQN [Mnih et al., 2015] with double Q-learning [van Hasselt et al., 2016, van Hasselt, 2010], prioritised replay [Schaul et al., 2016], dueling networks [Wang et al., 2016b], multi-step targets [Sutton, 1988, Sutton and…”
DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction (2020, background): “[3] Hado van Hasselt.”
Striving for Simplicity in Off-policy Deep Reinforcement Learning (2019, background): “Rainbow [22] suggests combining the improvement of different techniques such as distributional RL, prioritized experience replay [48], double Q-learning [59, 21], dueling architecture [60], etc ., into a single off-policy agent.”
Deep Reinforcement Learning and the Deadly Triad (2018, background): “This is known as double Q-learning (van Hasselt, 2010).”
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, methodology): “Q-learning with deep neural network function approximators provides a simple and practical scheme for RL with image observations, and is amenable to straightforward parallelization.”