Toward a modern theory of adaptive networks: expectation and prediction.

By Richard S. Sutton, Andrew Barto. 1981

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

RUDDER: Return Decomposition for Delayed Rewards (2018, background): “The next theorem states that TD has an exponential decay for Q-value updates even for eligibility traces [56, 6, 106, 99].”
Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type (1998, methodology): “Although the most commonly used ones are based on the statistical properties of presynaptic and postsynaptic activity (e.g., activity product, activity covariance, etc.) without considering the detailed temporal structure of the spike patterns, a number of formulations have indeed incorporated a fac”
Policy Gradient Coagent Networks (2011, background): “Methods for solving sequential decision problems with delayed reward, where the problems are formulated as Markov decision processes (MDPs), have been compared to the learning mechanisms of animal brains [3, 4, 9, 10, 13, 20, 22].”
Purkinje cell activity during classical conditioning with different conditional stimuli explains central tenet of Rescorla–Wagner model (2015, methodology): “Moreover, in contrast to the Hebbian model, it is a prime example of an error correction process in which behavioral changes result from violation of expectations (2).”
Tonic dopamine: opportunity costs and the control of response vigor (2007, unknown): “Not only does RL have a sound mathematical basis in the engineering theory of dynamic programming (Bertsekas and Tsitsiklis 1996), it also has long had a very close relation with psychological accounts of behavioral learning (Sutton and Barto 1981).”
Getting formal with dopamine and reward. (2002, methodology): “Neurons in the striatum and orbitofrontal cortex show models (Sutton and Barto, 1981).”
Evaluating the TD model of classical conditioning (2012, background): “Although an obvious simplification, this approach, in combination with an appropriate learning rule, can accomplish a surprisingly wide range of real-time learning phenomena (Sutton & Barto, 1981, 1990).”
A Connectionist Model of Brand–Quality Associations (2000, background): “It has also been assumed that the updating of connection weights will decrease with the number of trials, which allows the connection weights to converge instead of oscillating infinitely (Sutton and Barto 1981).”
Hippocampal mediation of stimulus representation: A computational theory (1993, background): “…extensive analyses at the behavioral level, resulting in powerful and elegant computational models that characterize the behavioral properties of this form of learning (Sutherland and Mackintosh, 1971; Rescorla and Wagner, 1972; Pearce and Hall, 1980; Sutton and Barto, 1981; Mackintosh, 1983).”
Timing rules for synaptic plasticity matched to behavioral function (2016, unknown): “For example, during feedback-based learning, a delayed error signal must selectivelymodify synapses active at the specific, earlier time when the neural command leading to an error was generated, a challenge known as the temporal credit assignment problem (Sutton and Barto, 1981).”
Game Theory of Mind (2008, background): “This learning could use dynamic programming [12], or Q-learning [13], or any biologically plausible scheme.”
Prospective Optimization (2014, methodology): “The temporal-differences algorithm in reinforcement learning is closely related to the Rescorla– Wagner model [3], [4], and approximates dynamic programming [5].”
Equivalence Relations: A Critical Analysis (2001, background): “When A and B are paired, their internal representatives are jointly activated, which strengthens an association between the two (e.g., Sutton & Barto, 1981).”