Temporal credit assignment in reinforcement learning

By Richard S. Sutton. 1984

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

An Actor-Critic Algorithm for Sequence Prediction (2016, methodology): “Our approach draws inspiration and borrows the terminology from the ﬁeld of reinforcement learning (RL) (Sutton & Barto, 1998), in particular from the actor-critic approach (Sutton, 1984; Sutton et al., 1999; Barto et al., 1983).”
ReasoNet: Learning to Stop Reading in Machine Comprehension (2016, background): “bT is called the reward baseline in the RL literature to lower the variance [23].”
Short-term plasticity as cause–effect hypothesis testing in distal reward learning (2014, background): “This problem was called the distal reward problem (Hull, 1943), or credit assignment problem (Sutton, 1984; Sutton and Barto, 1998).”
Reward-estimation variance elimination in sequential decision processes (2018, background): “This is a general problem, also known as the credit assignment problem [17].”
Batch Policy Gradient Methods for Improving Neural Conversation Models (2017, methodology): “Instead of directly using the re-weighted rewards, we will use the so called λ –return which is a convex combination of the re-weighted rewards and the value function (Sutton, 1988; 1984).”
Reinforcement Learning: A Survey (1996, methodology): “They can also begeneralized to real-valued reward through reward comparison methods (Sutton, 1984).”
The Option-Critic Architecture (2016, methodology): “We refer to the resulting system as an option-critic architecture, in reference to the actor-critic architectures (Sutton 1984).”
The Optimal Reward Baseline for Gradient-Based Reinforcement Learning (2001, background): “3 (cid:10)(cid:30)(cid:11) *( L (cid:28)SOT) (cid:20) ( M (cid:26)(cid:27)3 (cid:10)(cid:6)(cid:11) * ( M (cid:28) proposed by (Sutton, 1984).”
Gated-Attention Architectures for Task-Oriented Language Grounding (2017, methodology): “The architecture of the policy learning module is specific to the learning paradigm: (1) Imitation Learning or (2) Reinforcement Learning.”
Policy Networks with Two-Stage Training for Dialogue Systems (2016, background): “Such algorithms are known in general as actor-critic algorithms, where theQ approximator is the critic and πθ is the actor (Sutton, 1984; Barto et al., 1990; Bhatnagar et al., 2009).”
Reinforcement Learning with Replacing Eligibility Traces (2005, methodology): “Systematic empirical studiesof eligibility traces in conjunction with TD methods were made by Sutton (1984),and theoretical results have been obtained by several authors (e.g., Dayan, 1992;Jaakkola, Jordan & Singh, 1994; Tsitsiklis, 1994; Dayan & Sejnowski, 1994; Sutton& Singh, 1994).”
Algorithms for Sequential Decision Making (1996, background): “Much of the content of this chapter is a recapitulation of work in the operationsresearch literature [126, 15, 44, 46, 68, 13] and the reinforcement-learning literature [153, 173, 10, 145].”
Time-Derivative Models of Pavlovian Reinforcement (1990, methodology): “This chapter presents a model of classical conditioning called the temporaldierence (TD) model . The TD model was originally developed as a neuronlike unit for use in adaptive networks (Sutton and Barto 1987; Sutton 1984; Barto, Sutton and Anderson 1983).”
Auto-teaching: networks that develop their own teaching input (1993, background): “Reinforcement paradigms (Sutton, 1984) supply only judgments of right or wrong.”
Prospective Optimization (2014, methodology): “The temporal-differences algorithm in reinforcement learning is closely related to the Rescorla– Wagner model [3], [4], and approximates dynamic programming [5].”
Solving Non-Markovian Control Tasks with Neuro-Evolution (1999, background): “(2) It embodies many essential aspects of a whole class of learning tasks that involve temporal credit assignment [Sutton, 1984].”
Problem solving with reinforcement learning (1995, methodology): “8 Temporal Di erence Learning Temporal di erence learning (Sutton 1988) is another incremental learning method that can be used to learn value function predictions. The algorithm is described in detail in the next chapter, but here a brief overview is given. To explain the concept behind temporal di”