Dopamine signals for reward value and risk: basic and recent data

By Wolfram Schultz. 2010

319 citations recorded by the source corpus.

BackgroundPrevious lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.MethodsThe described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.ResultsDopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.ConclusionsNeurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

Theory of Mind: A Neural Prediction Problem (2013, neutral): “The central idea of “predictive coding” is that (some) neural responses contain information not about the value of a currently perceived stimulus, but about the difference between the stimulus value and the expected value (Fiorillo et al., 2003; Schultz et al., 1997; Schultz, 2010).”
Reward, dopamine and the control of food intake: implications for obesity (2010, unknown): “the smell of food), which is then processed as a predictor of reward (becoming a cue that is conditioned to the reward) [30,31]; the DA signal in response to the cue then serves to convey a ‘reward prediction error’ [31].”
Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. (2012, unknown): “…release encode motivational value and/or salience (Bromberg-Martin et al., 2010; Jin and Costa, 2010; Phillips et al., 2003; Redgrave et al., 2008; Schultz, 2010; Tsai et al., 2009) and regulate long-term changes in striatal synaptic plasticity (Owesson-White et al., 2008; Surmeier et al., 2009)…”
Perceptual learning, roving and the unsupervised bias. (2012, unknown): “For this reason, we propose that the critic has a much lower resolution and is therefore not located in the visual system but rather in the limbic and dopaminergic systems (Schultz, 2007, 2010; Schultz, Dayan, & Montague, 1997).”
Drosophila Learn Opposing Components of a Compound Food Stimulus (2014, background): “Dopaminergic neurons provide value signals in mammals and insects [1–3].”
FROM REINFORCEMENT LEARNING MODELS OF THE BASAL GANGLIA TO THE PATHOPHYSIOLOGY OF PSYCHIATRIC AND NEUROLOGICAL DISORDERS (2011, background): “Although the majority of dopamine neurons burst to positive prediction errors, smaller proportions of these neurons also burst under other condition”
The mysterious motivational functions of mesolimbic dopamine. (2012, unknown): “In describing causal factors that control behavior, the German philosopher Schopenhauer (1999) discussed the concept of motivation in relation to the way that organisms must be in a position to ''choose, seize, and even seek out the means of satisfaction.''”
Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. (2013, background): “It is known that the amplitude of the phasic dopamine signal declines as the stimulus becomes predicted (see for example, Schultz, 2010).”
GABA Neurons of the VTA Drive Conditioned Place Aversion (2012, background): “This mirrors the established role of the VTA in reward processing (Fields et al., 2007; Schultz, 2010).”
Peers increase adolescent risk taking by enhancing activity in the brain’s reward circuitry (2011, unknown): “Regions in this brain pathway (especially the NAcc), which is implicated in the valuation and prediction of potential rewards (Breiter & Rosen, 1999; Delgado, 2007; O’Doherty, 2004; Schultz, 2010), have been found in several recent functional neuroimaging studies to show especially heightened activa”
A Spiking Neural Model for Stable Reinforcement of Synapses Based on Multiple Distal Rewards (2013, result): “…as Fr´emaux et al. (2010) argued, ARG-STDP might be the favored over R-Max because it is the spirit of temporal difference learning (Sutton & Barto, 1998) and ARG-STDP agrees with the widespread interpretation of subcortical dopamine signals as a reward-prediction-error (Schultz, 2007, 2010).”
A visual analytics approach to understanding cycling behaviour (2012, unknown): “An introduction to C-IL 2 P is also presented, followed by a review of propositionalization and feature selection.”