True Online TD(lambda)

By Harm van Seijen, Richard S. Sutton. 2014

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learning (2015, methodology): “Extensions of the emphasis idea to action-value and control methods such as Sarsa(λ) and Q(λ), to trueonline forms (van Seijen & Sutton 2014), and to weighted importance sampling (Mahmood et al. 2014, in preparation) are also natural and remain for future work.”
Two-Timescale Networks for Nonlinear Value Function Approximation (2019, background): “Additionally, there have been a variety of algorithms derived for the linear setting, both for on-policy and off-policy learning (Sutton et al., 2009; Maei, 2011; van Seijen and Sutton, 2014; van Hasselt et al., 2014; Mahadevan et al., 2014; Sutton et al., 2016; Mahmood et al., 2017).”
Investigating Practical Linear Temporal Difference Learning (2016, methodology): “The proposed trueonline TD (TO-TD) prediction algorithm [24], and trueonline GTD (TO-GTD) prediction algorithm [23] remedy this issue, and have been shown to outperform conventional TD and gradient TD methods respectively on chain domains.”
Beyond the One Step Greedy Approach in Reinforcement Learning (2018, methodology): “We stress that the term “ n -step return” often refered to in the literature, is used in the context of policy evaluation (Sutton & Barto, 1998; Seijen & Sutton, 2014).”
A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning (2016, background): “rk opens up many avenues for eciently selecting , by providing a concrete greedy objective. One important direction is to extend the above analysis to use the recently introduced true-online traces [32]; here we focused on the more well-understood -return, but the proposed objective is not restricte”
A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximation (2018, background): “As the name suggests, the components of z 0: t roughly capture the extent to which each feature is eligible for receiving credit or blame for an observed TD error [Sutton and Barto, 1998, Seijen and Sutton, 2014].”