Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning

By John P. O'Doherty, Peter S. Dayan, J. Schultz, Ralf Deichmann, Karl J. Friston, R. J. Dolan. Science, 2004

2,124 citations recorded by the source corpus.

Instrumental conditioning studies how animals and humans choose actions appropriate to the affective structure of an environment. According to recent reinforcement learning models, two distinct components are involved: a “critic,” which learns to predict future reward, and an “actor,” which maintains information about the rewarding outcomes of actions to enable better ones to be chosen more frequently. We scanned human participants with functional magnetic resonance imaging while they engaged in instrumental conditioning. Our results suggest partly dissociable contributions of the ventral and dorsal striatum, with the former corresponding to the critic and the latter corresponding to the actor.

Subfields: Cognitive Neuroscience, Neurology

View Paper: https://www.semanticscholar.org/paper/45d3a2454ae38657c38429d2abbb3e65898d43cf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Neurology

Citations in the corpus

A specific role for posterior dorsolateral striatum in human habit learning (2009, result): “These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.”
Perceptions of moral character modulate the neural systems of reward during the trust game (2005, background): “This has led to the hypothesis that the caudate nucleus may serve as a key component of an 'actor-critic' model processing the contingent behavior that led to the feedback, with the purpose of guiding future action”
Environmental statistics and the trade-off between model-based and TD learning in humans (2011, methodology): “R habits correspond well to the policies learned by TD methods such as actor/critic [14, 15], and rule-based cognitive planning strategies seem to mirror model-based algorithms.”
Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning (2012, background): “Unlike the RL prediction error, which has been extensively studied (16, 17), the belief prediction error corresponds to the discrepancy be-tween the observed and expected frequency of actions of an opponent weighted by the associated payoffs.”
Behavioral / Systems / Cognitive Striatal Prediction Error Modulates Cortical Coupling (2010, positive): “In line with numerous previous studies (McClure et al., 2003; O’Doherty et al., 2004; Pessiglione et al., 2006; Jensen et al., 2007; den Ouden et al., 2009), the BOLD response in the putamen reflected prediction errors.”
Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package (2016, neutral): “…O’Doherty, Dayan, Seymour, & Dolan, 2006; Gläscher, Hampton, & O’Doherty, 2009; Hampton, Bossaerts, & O’Doherty, 2006; Iglesias et al., 2013; Li, Schiller, Schoenbaum, Phelps, & Daw, 2011; Mars et al., 2008; O’Doherty et al., 2004; O’Doherty, Hampton, & Kim, 2007; Xiang, Lohrenz, & Montague, 2013).”
A Dual Role for Prediction Error in Associative Learning (2008, positive): “Because previous studies demonstrated the role of the striatum and the prefrontal cortex in associative learning (e.g., Fletcher et al. 2001; O’Doherty et al. 2004; Corlett et al. 2004), we performed an additional restricted search in these areas, using anatomical masks generated from the PickAtlas…”