On-line Q-learning using connectionist systems

By Gavin Adrian Rummery, Mahesan Niranjan. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Asynchronous Methods for Deep Reinforcement Learning (2016, methodology): “The target value used by one-step Sarsa is r + γQ ( s (cid:48) , a (cid:48) ; θ − ) where a (cid:48) is the action taken in state s (cid:48) (Rummery & Niranjan, 1994; Sutton & Barto, 1998).”
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (2017, background): “Sarsa(λ) (Rummery & Niranjan, 1994) is a control algorithm that learns an approximate action-value function of a sequence of improving policies.”
Deep learning in neural networks: An overview (2014, background): “Many variants of traditional RL exist (e.g., Barto et al., 1983; Watkins, 1989; Watkins and Dayan, 1992; Moore and Atkeson, 1993; Schwartz, 1993; Baird, 1994; Rummery and Niranjan, 1994; Singh, 1994; Baird, 1995; Kaelbling et al., 1995; Peng and Williams, 1996; Mahadevan, 1996; Tsitsiklis and van Ro”
A Survey on Neural Architecture Search (2019, methodology): “Temporal Difference Learning Approaches like SARSA (Rummery and Niranjan, 1994), TD-λ (Sutton and Barto, 1998), and Q-learning (Watkins, 1989) attempt to find this policy implicitly via approximating the optimal value functions.”
RUDDER: Return Decomposition for Delayed Rewards (2018, methodology): “For look-back advice, we use SARSA [97] instead of Q-learning as suggested by the authors.”
Meta-Gradient Reinforcement Learning (2018, methodology): “…of deep reinforcement learning updates are differentiable functions of the return, including, for instance, value-based methods like SARSA( λ ) [Rummery and Niranjan, 1994, Sutton and Barto, 2018] and DQN [Mnih et al., 2015], policy-gradient methods [Williams, 1992], or actor-critic algorithms…”
Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems (2020, methodology): “In this section, we study the effect of replacing the RGA with the classical sequential quadratic programming (SQP) optimization algorithm in the lower level of the overall bilevel algorithm for obtaining NLDT OL .”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, methodology): “On-policy methods such as SARSA [35], estimate the value of a policy while using the same policy for control.”
Smoothed Dual Embedding Control (2017, background): “The resulting algorithm resembles to SARSA with particular policy.”
A General Family of Robust Stochastic Operators for Reinforcement Learning (2018, methodology): “It is important to note that our approach can be extended to variants of the Bellman operator such as SARSA [15], policy evaluation [17] and fitted Q-iteration [11].”
Distributional Reinforcement Learning with Quantile Regression (2017, methodology): “Because knowing this value for the optimal policy is sufﬁcient to act optimally, it is the object modelled by classic value-based meth-ods such as SARSA (Rummery and Niranjan 1994) and Q-Learning (Watkins and Dayan 1992), which use Bellman’s equation (Bellman 1957) to efﬁciently reason about value.”
Increasing the Action Gap: New Operators for Reinforcement Learning (2015, methodology): “Weemphasize that while we focus on the Bellman operator, our results easily extend to its variations such as SARSA (Rummery and Niranjan 1994), policy evaluation (Sutton 1988), and ﬁtted Q-iteration (Ernst, Geurts, and We-henkel 2005).”
Regularized Behavior Value Estimation (2021, methodology): “We start by drawing inspiration from the SARSA up-date (Rummery and Niranjan, 1994; Van Seijen et al., 2009).”