Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task

By Wolfram Schultz, P. Apicella, T. Ljungberg. 1993

The present investigation had two aims: (1) to study responses of dopamine neurons to stimuli with attentional and motivational significance during several steps of learning a behavioral task, and (2) to study the activity of dopamine neurons during the performance of cognitive tasks known to be impaired after lesions of these neurons. Monkeys that had previously learned a simple reaction time task were trained to perform a spatial delayed response task via two intermediate tasks. During the learning of each new task, a total of 25% of 76 dopamine neurons showed phasic responses to the delivery of primary liquid reward, whereas only 9% of 163 neurons responded to this event once task performance was established. This produced an average population response during but not after learning of each task. Reward responses during learning were significantly more numerous and pronounced in area A10, as compared to areas A8 and A9. Dopamine neurons also showed phasic responses to the two conditioned stimuli. These were the instruction cue, which was the first stimulus in each trial and indicated the target of the upcoming arm movement (58% of 76 neurons during and 44% of 163 neurons after learning), and the trigger stimulus, which was a conditioned incentive stimulus predicting reward and eliciting a saccadic eye movement and an arm reaching movement (38% of neurons during and 40% after learning). None of the dopamine neurons showed sustained activity in the delay between the instruction and trigger stimuli that would resemble the activity of neurons in dopamine terminal areas, such as the striatum and frontal cortex. Thus, dopamine neurons respond phasically to alerting external stimuli with behavioral significance whose detection is crucial for learning and performing delayed response tasks. The lack of sustained activity suggests that dopamine neurons do not encode representational processes, such as working memory, expectation of external stimuli or reward, or preparation of movement. Rather, dopamine neurons are involved with transient changes of impulse activity in basic attentional and motivational processes underlying learning and cognitive behavior.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, neutral): “Learning by reward in monkeys was linked to dopaminergic activity during the 1990s with studies by Schultz et al. (1993, 1997); Schultz (1998).”
Dysfunction of ventral striatal reward prediction in schizophrenia (2006, background): “Reward-indicating stimuli have been associated with a phasic short-term increase in the firing rate of dopaminergic neurons (Grace, 1991; Schultz et al., 1993).”
Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis (2011, background): “(27) first observed these responses, they hypothesized that “dopamine neurons are involved with transient changes of impulse activity in basic attentional and motivational processes underlying learning and cognitive behavior” (27).”
Reward-motivated learning: mesolimbic activation precedes memory formation. (2006, methodology): “However, given the extensive electrophysiological data documenting the responsiveness of dopaminergic midbrain neurons to reward cues (Schultz et al., 1993, 1997), it seems unlikely that nondopaminergic midbrain cells would respond to reward cues without concomitant responses in dopamine neurons.”
The mysterious motivational functions of mesolimbic dopamine. (2012, unknown): “…have shown that presentation of novel or unexpected food reinforcers is accompanied by transient increases in the activity of putative ventral tegmental DA neurons, but that this effect goes away with regular presentation, or repeated exposure through training (Schultz et al., 1993; Schultz, 2010).”
A Neural Computational Model of Incentive Salience (2009, methodology): “In studies to tease apart ‘wanting’ from ‘liking’ and from learned predictions of reward, we used a simple, serial paradigm containing two CSs that predicted a sucrose UCS [13, 75 ].”
Dopamine signals for reward value and risk: basic and recent data (2010, background): “In particular, other strong attention generating events such as reward omission, conditioned inhibitors and aversive stimuli induce predominantly depressions and rarely genuine dopamine activations [14,29].”
Tonic dopamine: opportunity costs and the control of response vigor (2007, unknown): “These recordings suggested that the phasic (bursting and pausing) spiking activity of dopamine cells reports to the striatum a specific “prediction error” signal (Ljungberg et al. 1992; Schultz et al. 1993; Schultz 1998; Waelti et al. 2001).”