Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

By Matthew R. Roesch, Donna J. Calu, Geoffrey Schoenbaum. Nature Neuroscience, 2007

613 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Molecular Biology, Social Psychology

View Paper: https://www.semanticscholar.org/paper/77c1f6f24afc317364e5e55f03f219bc5cc967ee

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Molecular Biology

In Social Psychology

Citations in the corpus

Updating dopamine reward signals (2013, background): “5 s compared to long, variable delays of 1–7 s [17].”
Dopamine signals for reward value and risk: basic and recent data (2010, background): “In choices between different reward values or delays, the dopamine responses to the presentation of choice options reflects the animal's future chosen reward [34] or the highest possible reward of two available choice options [35].”
Computational models of reinforcement learning: the role of dopamine as a reward signal (2010, background): “…groups have correlated the activity of DA neurons with the TD error originally proposed by the algorithm (Bayer and Glimcher 2005; Morris et al. 2004; Montague et al. 1996; Nakahara et al. 2004; Roesch et al. 2007; Schultz et al. 1997) and found that it indeed followed the model at many levels.”
Dialogues on prediction errors. (2008, neutral): “Researchers have only recently started to examine which one of these models, if any, is implemented in the brain, and results so far are equivocal [ 23 ,59].”
Cognitive Control (2012, neutral): “Recent evidence looking primarily at one dopaminergic nucleus seems to support SARSA [78], whereas evidence from a rodent study of the other major dopaminergic nucleus favors Q -learning [49], [79].”
The Neurobiology of Decision: Consensus and Controversy (2009, positive): “In a similar manner, if different cues predict rewards after different delays, the cue-elicited response decreases as the delay-to-reward increases, consistent with a prediction that incorporates discounting of future rewards (Fiorillo et al., 2008; Kobayashi and Schultz, 2008; Roesch et al., 2007).”
Midbrain dopamine neurons signal preference for advance information about upcoming rewards (2009, background): “Consistent with previous studies of reward coding (Schultz and Romo, 1990; Kawagoe et al., 2004; Roesch et al., 2007; Matsumoto and Hikosaka, 2007), we observed similar neural coding of behav-”
Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding (2011, background): “Each aspect of this triad confounds the role of the phasic DA bursts in the flexible acquisition of instrumental values [36–42].”
Reinforcement learning in the brain (2009, background): “Interestingly, recent electrophysiological recordings in non-human primates (Morris et al., 2006) and in rats (Roesch et al., 2007) suggest that dopaminergic neurons in the brain may indeed be conveying a prediction error that is based on state-action values (rather than state values, as in the…”
Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis (2011, background): “tence of a reward prediction error-like signal in midbrain dopamine neurons of that species (62).”
Choice, uncertainty and value in prefrontal and cingulate cortex (2008, background): “When an animal is presented with a conditioned stimulus that predicts a later reward, phasic responses are induced in dopamine neurons that encode the expectation at the time of the cue and encode a quantitative prediction error when the outcome is observe”
Decision making in recurrent neuronal circuits. (2008, background): “Consistent with a prediction error signal, spiking activity of dopamine neurons increases with both reward magnitude and probability (Fiorillo et al., 2003; Tobler et al., 2005; Roesch et al., 2007).”
Hippocampal replays under the scrutiny of reinforcement learning models. (2018, methodology): “MF-RL models were later successfully applied to an extended corpus of experimental data, including instrumental conditioning in rodents (Morris et al. 2006; Roesch et al. 2007) but also instrumental learning tasks in humans, through the use of model-based functional MRI data analysis approaches (O’D”
Reinforcement learning: the good, the bad and the ugly. (2008, neutral): “when taking together [56 ,58 ]).”
Model-Based and Model-Free Pavlovian Reward Learning: Revaluation, Revision and Revelation (2014, methodology): “One view suggests that a phasic dopamine pulse is the key teaching signal for model-free prediction and action learning, as in one of reinforcement learning’s model-free learning methods: the actor critic (Barto, Sutton, & Anderson, 1983), Q-learning (Roesch et al., 2007; Watkins, 1989), or SARSA (M”