Learning from delayed rewards

By Ben J. A. Kröse. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Decision Transformer: Reinforcement Learning via Sequence Modeling (2021, methodology): “Some prior methods for reinforcement learning bear more resemblance to static supervised learning, such as Q-learning (Watkins, 1989; Mnih et al., 2013), which still uses iterative backups, or likelihood-based methods such as behavior cloning, which do not (discussed in previous section).”
Addressing Function Approximation Error in Actor-Critic Methods (2018, methodology): “In Q-learning, the value function can be learned using temporal difference learning (Sutton, 1988; Watkins, 1989), an update rule based on the Bellman equation (Bellman, 1957).”
Trust-PCL: An Off-Policy Trust Region Method for Continuous Control (2017, methodology): “More recently, Haarnoja et al. (2017) showed that soft Q-learning (a single-step special case of PCL) can succeed on more challenging environments, such as a variant of the Swimmer task we consider below.”
Bridging the Gap Between Value and Policy Based Reinforcement Learning (2017, methodology): “We begin the formulation by reviewing the key elements of Q-learning [43, 44], which uses a notion of hard-max Bellman backup to enable off-policy TD control.”
PGQ: Combining policy gradient and Q-learning (2016, methodology): “In Q-learning (Watkins, 1989) , which would be equivalent to using an optimizing critic that bootstraps using the max Q-value at the next state, i.e. , In REINFORCE the critic is the Monte Carlo return from that state on, If the return trace is truncated and a bootstrap is performed after n -steps,…”
When to use parametric models in reinforcement learning? (2019, methodology): “Although SimPLe used limited data, it used a large number of samples from the model, similar to using P = 800,000.2 Rainbow DQN One of the main results by Kaiser et al. [2019] was to compare SimPLe to Rainbow DQN [Hessel et al., 2018a], which combines the DQN algorithm [Mnih et al., 2013, 2015] with”
Reinforcement Learning with Unsupervised Auxiliary Tasks (2016, methodology): “The actionvalue function Qπ(s, a) = E [Rt:∞|st = s, at = a, π] is the expected return following action a from state s. Value-based reinforcement learning algorithms, such as Q-learning (Watkins, 1989), or its deep learning instantiations DQN (Mnih et al., 2015) and asynchronous Q-learning (Mnih et a”
Deep Reinforcement Learning with Double Q-Learning (2015, background): “Estimates for the optimal action values can be learned using Q-learning (Watkins, 1989), a form of temporal difference learning (Sutton, 1988).”
Safe and Efficient Off-Policy Reinforcement Learning (2016, methodology): “Q k (i.e. ε k = 0 ), we deduce that Watkins’ Q( λ ) (e.g., Watkins, 1989; Sutton and Barto, 1998) converges a.s. to Q ∗ (under the assumption that µ k commutes asymptotically with the greedy policies, which is satisﬁed for e.g. µ k deﬁned by (8)).”
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, methodology): “Independent Q-learning (Tan, 1993) trains independent action-value functions for each agent using Q -learning (Watkins, 1989).”
Deep learning in neural networks: An overview (2014, methodology): “…Williams, 1996; Prokhorov &Wunsch, 1997; Rummery &Niranjan, 1994; Santamaría, Sutton, & Ram, 1997; Schwartz, 1993; Singh, 1994; Sutton & Barto, 1998; Sutton, Szepesvári, & Maei, 2008; Tsitsiklis & van Roy, 1996; van Hasselt, 2012; Watkins, 1989; Watkins & Dayan, 1992; Wiering & Schmidhuber, 1998b).”
A Survey on Neural Architecture Search (2019, methodology): “Zhong et al. (2018) also use a Q-learning-based algorithm to search for network architectures.”
AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “Following Block-QNN [54], which applies a variant form of Bellman’s Equation [50], each transition in an episode is ( s t , a t , R, s t +1 ), where R is the reward after the network is compressed.”
Learning to Prune Filters in Convolutional Neural Networks (2018, neutral): “[3] explored with more possible architectures using Q-learning [31, 26].”
Designing Neural Network Architectures using Reinforcement Learning (2016, methodology): “For instance, methods using CNNs to approximate the Q -learning utility function (Watkins, 1989) have been successful in game-playing agents (Mnih et al., 2015; Silver et al., 2016) and robotic control (Lillicrap et al., 2015; Levine et al., 2016).”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, methodology): “Deep Q-Networks (DQN) [22] incorporates a variant of the Q-learning algorithm [23], by using deep neural networks (DNNs) as a non-linear Q function approximator over high-dimensional state spaces (e.g. the pixels in a frame of an Atari game).”
Unmasking Clever Hans predictors and assessing what machines really learn (2019, methodology): “Using Q-learning [113], a model-free reinforcement learning technique, a network has been trained to fit an action-value function.”