Practical Issues in Temporal Difference Learning

By G. Tesauro. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement Learning with Replacing Eligibility Traces (2005, methodology): “The second mechanism that has been widely used for handling delay is the el-igibility trace.1 Introduced by Klopf (1972), eligibility traces have been used in avariety of reinforcement learning systems (e.g., Barto, Sutton & Anderson, 1983;Lin, 1992; Tesauro, 1992; Peng & Williams, 1994).”
Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning (2016, background): “x for a tabular implementation and require the use of function approximation. One of the most impressive early works, bringing together RL and neural networks presented a worldclass backgammon agent (Tesauro, 1992). Nevertheless, combined directly, divergence was shown even on simple problems (Boyan”
Large-scale kernel machines (2007, background): “Until very recently, empirical studies often found that deep networks generally performed no better, and often worse, than neural networks with one or two hidden layers [Tesauro, 1992].”
Proto-value Functions: A Laplacian Framework for Learning Representation and Control in Markov Decision Processes (2007, methodology): “Parametric approaches using linear architectures, such as radial basis functions (Lagoudakis and Parr, 2003), and nonlinear architectures, such as neural networks (Tesauro, 1992), have been extensively explored.”
Neural Networks and Deep Learning (2018, unknown): “Early methods for using neural networks in reinforcement learning were proposed in [296, 349, 492, 493, 494].”
Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method (2005, background): “Although a lot of successful applications exist [ Tes92 , Lin92, Rie00], also a lot of problems have been reported [BM95].”
DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess (2016, unknown): “Reinforcement learning has been successfully applied in backgammon [16] and checkers [13].”
Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. (2003, background): “The modules would exchange many reinforcement signals, in a ward, and techniques for learning the value function, such as temporal difference learning (Tesauro, 1992). manner that is more decentralized than the way neuro-modulatory systems are thought to operate.”
Reinforcement Learning and Its Relationship to Supervised Learning (2004, background): “This was observed in the TD-gammon system, where human experts changed the way they play certain backgammon positions after studying the policy learned by reinforcement learning (Tesauro, 1992, 1994, 1995).”
Fast Online Q(λ) (1998, unknown): “It combines Q-learning (Watkins, 1989; Watkins & Dayan, 1992) and TD(λ) (Sutton, 1988; Tesauro, 1992).”
A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning (2016, background): “For example, in off-policy learning (e.g., learning from demonstrations), large importance sampling ratios can destabilize provably convergent gradient temporal difference learning methods, when the parameters are not set in a very particular way (λ = 0) [33].”
Accelerated Neural Evolution through Cooperatively Coevolved Synapses (2008, background): “This kind of approach has been studied widely with success in tasks such as pole-balancing (Lin and Mitchell, 1992), pursuit-evasion games (Lin, 1992), and backgammon (Tesauro, 1992).”