Surprise signals in anterior cingulate cortex: Neuronal encoding of unsigned reward prediction errors driving adjustment in behavior

By Benjamin Y. Hayden, Sarah R. Heilbronner, John M. Pearson, M. Platt. 2011

390 citations recorded by the source corpus.

In attentional models of learning, associations between actions and subsequent rewards are stronger when outcomes are surprising, regardless of their valence. Despite the behavioral evidence that surprising outcomes drive learning, neural correlates of unsigned reward prediction errors remain elusive. Here we show that in a probabilistic choice task, trial-to-trial variations in preference track outcome surprisingness. Concordant with this behavioral pattern, responses of neurons in macaque (Macaca mulatta) dorsal anterior cingulate cortex (dACC) to both large and small rewards were enhanced when the outcome was surprising. Moreover, when, on some trials, probabilities were hidden, neuronal responses to rewards were reduced, consistent with the idea that the absence of clear expectations diminishes surprise. These patterns are inconsistent with the idea that dACC neurons track signed errors in reward prediction, as dopamine neurons do. Our results also indicate that dACC neurons do not signal conflict. In the context of other studies of dACC function, these results suggest a link between reward-related modulations in dACC activity and attention and motor control processes involved in behavioral adjustment. More speculatively, these data point to a harmonious integration between reward and learning accounts of ACC function on one hand, and attention and cognitive control accounts on the other.

Subfields: Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Theory of Mind: A Neural Prediction Problem (2013, neutral): “…distributed throughout the cortex and respond to specific stimulus features (Meyer and Olson, 2011; den Ouden et al., 2012), rather than, for example, a single “error region” signaling the overall amount of error or degree to which the observed stimulus is unpredicted (e.g., Hayden et al., 2011).”
Neurophysiology of performance monitoring and adaptive behavior. (2014, background): “This is in line with recordings from the aMCC homolog in monkeys showing neuronal activity coding valence-independent prediction errors (227, 346) and novelty of stimuli (347).”
Error awareness and the error-related negativity: evaluating the first decade of evidence (2012, background): “…of an error on a given trial (Brown and Braver, 2005), or signaling the unsigned reward prediction error, or “surprise” of a given response (Alexander and Brown, 2011; Hayden et al., 2011), it can be said with certainty that this activity differs with respect to subjective error awareness.”
The foraging brain (2015, background): “The dorsal anterior cingulate cortex (dACC) has been a focus of studies of the neural basis of foraging because it is a central node in the decision-making network — it integrates information from reward and associative regions and generates a control signal that influences selection of actions [31–”
Neuronal Reward and Decision Signals: From Theories to Data. (2015, background): “Unpredictedness enhances existing reward responses in anterior cingulate cortex (206).”
The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization (2014, background): “Other theories (Courville et al., 2006; Hayden et al., 2011; Pearce and Hall, 1980) focus on the aspect that learning is mainly driven by the surprisingness of an event, regardless of its valence, which can be characterized by absolute RPEs (|RPEs|).”