Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans

By Mathias Pessiglione, Ben Seymour, Guillaume Flandin, R. J. Dolan, ChrisD . Frith. Nature, 2006

1,566 citations recorded by the source corpus.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-l-phenylalanine; l-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with l-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/91a0b19076123facd721502f546fac76120cb5af

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Molecular Biology

Citations in the corpus

Spatial Cognition and the Brain (2008, background): “This simple and elegant model has provided a quantitative description of a great variety of behavioral data on learning and has received strong support from evidence that the prediction error signal can be seen in the striatal targets of the dopamine system (O’Doherty et al. 2004; Pessiglione et al.”
Hippocampal replays under the scrutiny of reinforcement learning models. (2018, methodology): “2004), showing reward prediction error correlates in the human basal ganglia (Bavard et al. 2018; Palminteri et al. 2015; Pessiglione et al. 2006).”
How Prediction Errors Shape Perception, Attention, and Motivation (2012, background): “…have also been reported in cortical areas, including in the orbitofrontal cortex to reward (Takahashi et al., 2009; Sul et al., 2010), the insular cortex to punishment (Pessiglione et al., 2006), and in the medial prefrontal cortex to both positive and negative feedback (Matsumoto et al., 2007).”
Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. (2012, background): “…Draganski et al. 2008), and in particular, evidence for a rostrocaudal organization from premotor/PFC to corresponding regions of striatum (Inase et al. 1999; Lehericy, Ducros, Krainik, et al. 2004; Lehericy, Ducros, Vande Moortele, et al. 2004; Postuma and Dagher 2006; Draganski et al. 2008).”
Affective neuroscience of pleasure: reward in humans and animals (2008, background): “R stamping-in or habit reinforcement (Day and Carelli 2007; Di Chiara and Bassareo 2007; Everitt and Robbins 2005; Fields et al. 2007; Hyman et al. 2006; Kelley 2004; Pessiglione et al. 2006; Schultz 2006; Schultz et al. 1997; Tobler et al. 2005; Wise 2006).”
Mechanisms of hierarchical reinforcement learning in cortico-striatal circuits 2: evidence from fMRI. (2012, background): “Many human reinforcement learning studies report RPE correlates across the extent of the striatum (McClure et al. 2003; O'Doherty et al. 2003; Pessiglione et al. 2006; Rutledge et al. 2010), a finding that we replicate here.”
Report Dopamine Enhances Expectation of Pleasure in Humans (2009, neutral): “, dihydroxy-L-phenylalanine; L-DOPA) have been shown to augment a striatal signal that expresses reward prediction errors during instrumental learning, thereby increasing the likelihood of choosing stimuli associated with greater monetary gains [5].”
From prediction error to incentive salience: mesolimbic computation of reward motivation (2012, background): “Also, dopamine manipulation by drugs have been reported to change learning and learned performance in people (Pessiglione et al., 2006; Moustafa et al., 2008; Samson et al., 2010).”
Generalization of value in reinforcement learning by humans (2012, result): “Although some results suggest that prediction errors in striatal BOLD signal may in part reflect dopaminergic inputs there (Pessiglione et al., 2006; Knutson & Gibbs, 2007; Schott et al., 2008; Schönberg et al., 2010), it is not possible to isolate the underlying neural cause for our effect, or in…”
Dopaminergic reward system: a short integrative review (2010, background): “A recent study on the other hand, showed faster learning as well as an increase in winning at gambling in response to DA consumption [ 9 ].”
The Influence of Affect on Beliefs, Preferences and Financial Decisions (2009, positive): “Recent work in economics (Elster (1998), Caplin and Leahy (2001)) has proposed that emotions may play a role in decision making under risk, following a signiﬁcant stream of work in the psychology literature (Loewenstein, Weber, Hsee, and Welch (2001)).”
Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia (2008, background): “Presently, reward and reinforcement are assigned to ventral striatum (Schultz et al., 1997; Pessiglione et al., 2006), the central parts of the basal ganglia play a role in cognitive functions (Levy et al.”
Reinforcement learning: the good, the bad and the ugly. (2008, neutral): “There is some indirect evidence [27–29,30 ] for the involvement of dopamine and dopaminergic mechanisms in learning from reward prediction errors in humans, along with more direct evidence from an experiment involving the pharmacological manipulation of dopamine [31 ].”
Processing of social and monetary rewards in the human striatum. (2008, background): “Its involvement in processing monetary reward in humans was also demonstrated in a study combining pharmacological and imaging techniques (Pessiglione et al., 2006).”
A Dual Role for Prediction Error in Associative Learning (2008, neutral): “…(key dopaminergic targets) has been shown to covary with both predictions and prediction errors (Fletcher et al. 2001; McClure et al. 2003; Corlett et al. 2004; O’Doherty et al. 2004; Seymour et al. 2004; Turner et al. 2004; Gläscher and Büchel 2005; Pessiglione et al. 2006; Jensen et al. 2007).”