Reinforcement learning with replacing eligibility traces

By Satinder Singh, Richard S. Sutton. 2004

259 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

RUDDER: Return Decomposition for Delayed Rewards (2018, background): “t;a t) (expected SARSA) : (A194) Temporal Difference with Eligibility Traces. Eligibility traces have been introduced to propagate back reward information of an episode and are now standard for TD() [119]. However, the eligibility traces are exponentially decaying when propagated back. The accumulat”
Learning to Act by Predicting the Future (2016, neutral): “The convergence of such methods was analyzed early on and they were seen as theoretically advantageous, particularly when function ap-proximators are used (Bertsekas, 1995; Sutton, 1995; Singh & Sutton, 1996).”
An Introduction to Intertask Transfer for Reinforcement Learning (2011, methodology): “There are many possible approaches to learning such a policy, but this article will focus ontemporal difference(TD) methods, such asQ-learning (Sutton 1988; Watkins 1989) andSarsa (Rummery & Niranjan 1994; Singh & Sutton 1996), which learn to approximateQ : S × A 7→ R.”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “based policy evaluation. Achieving the same linear rate asWai et al.(2018), three advantages are claimed inCassano et al.(2018): i) data-independent memory requirement; ii) use of eligibility traces (Singh and Sutton, 1996); iii) no need for synchronization in sampling. More recently, standard TD le”
Free-Energy Based Reinforcement Learning for Vision-Based Navigation with High-Dimensional Sensory Inputs (2010, methodology): “In this study, we use truncated replacing traces [ 8 ] which are defined as”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, methodology): “1998). Nevertheless, extended eligibility traces across multiple time steps are considered convenient heuristic tools to speed up learning in temporal difference algorithms such as TD() or SARSA() (Singh and Sutton, 1996; Sutton and Barto, 1998). In policy gradient methods (Williams, 1992) as well a”
Reinforcement learning for factored Markov decision processes (2002, result): “These results are consistent with previous results on learning with eligibility traces [Singh and Sutton 1996; Singh and Dayan 1998; He and Shayman 1999], although the specific best value for λ varies for different problems in the literature.”
TD_gamma: Re-evaluating Complex Backups in Temporal Difference Learning (2011, methodology): “While several different algorithms exist within the TD(λ) family—the original incremental and online algorithm [2], replacing traces [3], least-squares algorithms [4], algorithms for learning state-action value functions [1, 5], and algorithms for adapting λ [6], among others—the λ-return formulatio”
Transfer Learning via Inter-Task Mappings for Temporal Difference Learning (2007, methodology): “TAYLOR, STONE AND LIU However, the basic unenhanced TD algorithms, such as Q-Learning (Watkins, 1989) and Sarsa (Rummery and Niranjan, 1994; Singh and Sutton, 1996), have been found slow to produce nearoptimal behaviors in practice.”
Cross-domain transfer for reinforcement learning (2007, background): “Keepers learn using Sarsa (Rummery & Niranjan, 1994; Singh & Sutton, 1996) for estimating the action-value function.”
Intrinsically motivated neuroevolution for vision-based reinforcement learning (2011, methodology): “VQ-NES was evaluated on a vision-based version of the (8) well-known mountain car RL benchmark [15,21], shown in”
Autonomous transfer for reinforcement learning (2008, methodology): “The CMAC is two-dimensional and has 14 tilings (repeating the setup detailed in Singh and Sutton [14]).”
Teacher-Student Framework: a Reinforcement Learning Approach (2014, background): “In the next three subsections, we recall the mountain car domain [14], a popular benchmark for reinforcement learning and give some details about the implementations of our two approaches.”
Affective Personalization of a Social Robot Tutor for Children's Second Language Skills (2016, methodology): “In order to achieve this, we used a standard SARSA algorithm Singh and Sutton (1996), where the reward was the weighted sum of the valence and engagement r = 0.”