Temporal difference models describe higher-order learning in humans

By Ben Seymour, John P. O'Doherty, Peter S. Dayan, M. Koltzenburg, Anthony K. Jones, R. J. Dolan, Karl J. Friston, Richard S. J. Frackowiak. Nature, 2004

627 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Physiology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/d3ea470c124ae4d7bac709be1da6bd31570efc64

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Physiology

In Social Psychology

Citations in the corpus

Environmental statistics and the trade-off between model-based and TD learning in humans (2011, methodology): “Intuitively, S–R habits correspond well to the policies learned by TD methods such as actor/critic [14, 15], and rule-based cognitive planning strategies seem to mirror model-based algorithms.”
Neural mechanisms of autonomic, affective, and cognitive integration (2005, background): “From Singer et al. (2004) and Seymour et al. (2004).”
A Dual Role for Prediction Error in Associative Learning (2008, positive): “…(key dopaminergic targets) has been shown to covary with both predictions and prediction errors (Fletcher et al. 2001; McClure et al. 2003; Corlett et al. 2004; O’Doherty et al. 2004; Seymour et al. 2004; Turner et al. 2004; Gläscher and Büchel 2005; Pessiglione et al. 2006; Jensen et al. 2007).”
Dopamine, uncertainty and TD learning (2005, positive): “Introduction There is an impressively large body of physiological, imaging, and psychopharmacological data regarding the phasic activity of dopaminergic (DA) cells in the midbrains of monkeys, rats and humans in classical and instrumental conditioning tasks involving predictions of future rewards [1”
A meta-analysis of instructed fear studies: Implications for conscious appraisal of threat (2010, background): “One may speculate that the anterior insula is involved in generating the subjective feeling of fear based on interoceptive input (Critchley et al., 2004) or participates in reinforcement learning (Seymour et al., 2004, 2005).”
How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis (2012, methodology): “Activity and plasticity in striatal neurons, a major target of dopaminergic efferents, are dynamically sensitive to these dopaminergic prediction error signals, which enable the striatum to represent RL values (O’Doherty et al. , 2004; Frank, 2005; Daw & Doya, 2006).”
Reinforcement learning in the brain (2009, background): “…afferents), and might not differentiate inhibitory from excitatory afferent activity, as is illustrated by the fact that BOLD correlates of positive prediction errors for pain and punishment have also been found in the striatum (Jensen et al., 2007;Menon et al., 2007; Seymour et al., 2004).”
Insights from the application of computational neuroimaging to social neuroscience. (2013, positive): “The computational approach to neuroimaging was brought to bear initially on the domains of value-based learning and decision-making in nonsocial situations, as a means of gaining insight into the computations performed by different brain regions in learning to predict rewarding and punishing outcome”
Anterior Insular Cortex and Emotional Awareness (2013, positive): “One study specifically showed that AIC encodes both prediction of pain and pain prediction error in the same task (Seymour et al., 2004), which directly supports the functional integration and predictive encoding account of AIC.”
The basal ganglia and the cerebellum: nodes in an integrated network (2018, neutral): “Part b is adapted from reF.(98), Macmillan Publishers Limited.”
Behavioral and neural properties of social reinforcement learning (2011, methodology): “Reaction time has been shown in previous studies to be a reliable indicator of learning contingencies and speeding or slowing in reaction times has been associated with conditioning as predicted by reinforcement learning models (Seymour et al., 2004; Bray and O’Doherty, 2007).”