Expectation and Surprise Determine Neural Population Responses in the Ventral Visual Stream

By Tobias Egner, Jim M. Monti, Christopher Summerfield. 2010

359 citations recorded by the source corpus.

Visual cortex is traditionally viewed as a hierarchy of neural feature detectors, with neural population responses being driven by bottom-up stimulus features. Conversely, “predictive coding” models propose that each stage of the visual hierarchy harbors two computationally distinct classes of processing unit: representational units that encode the conditional probability of a stimulus and provide predictions to the next lower level; and error units that encode the mismatch between predictions and bottom-up evidence, and forward prediction error to the next higher level. Predictive coding therefore suggests that neural population responses in category-selective visual regions, like the fusiform face area (FFA), reflect a summation of activity related to prediction (“face expectation”) and prediction error (“face surprise”), rather than a homogenous feature detection response. We tested the rival hypotheses of the feature detection and predictive coding models by collecting functional magnetic resonance imaging data from the FFA while independently varying both stimulus features (faces vs houses) and subjects' perceptual expectations regarding those features (low vs medium vs high face expectation). The effects of stimulus and expectation factors interacted, whereby FFA activity elicited by face and house stimuli was indistinguishable under high face expectation and maximally differentiated under low face expectation. Using computational modeling, we show that these data can be explained by predictive coding but not by feature detection models, even when the latter are augmented with attentional mechanisms. Thus, population responses in the ventral visual stream appear to be determined by feature expectation and surprise rather than by stimulus features per se.

Subfields: Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Theory of Mind: A Neural Prediction Problem (2013, positive): “Other experiments show that a tone can be used as a cue to predict the orientation of an upcoming grating (Kok et al., 2012a), and either a colored frame or an auditory tone can predict whether an upcoming image will be a face or a house (den Ouden et al., 2010; Egner et al., 2010).”
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning (2016, methodology): “Our work here builds on previous work in next-frame video prediction [26, 28, 29, 38, 48], but we take particular inspiration from the concept of “predictive coding” from the neuroscience literature [2, 5, 7, 9, 11, 21, 25, 32, 39, 40, 47, 49].”
A ventral visual stream reading center independent of visual experience. (2011, neutral): “For instance, it has been shown [29] that the activation of the fusiform face area, an equivalent of the VWFA specialized for face perception, depends on subjects’ expectations and on the mismatch between those expectations and the actual sensory input, but not on whether stimuli actually depict fac”
How Do Expectations Shape Perception? (2018, background): “Such studies showed for instance that expectation suppression in macaque IT was associated with decreased decoding performance [66]; that BOLD responses in posterior superior temporal sulcus (pSTS) to expected words were weaker and contained less information, as quanti ﬁ ed by representational simil”
Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study (2011, background): “Indeed, a recent study suggests that the population response to expected and unexpected events may best be explained by a combination of prediction- and prediction-error-related responses (Egner et al., 2010).”
Predictive Processing: A Canonical Cortical Computation (2018, background): “This is indeed the case when stimuli become predictable either as the result of a learned association with a preceding stimulus (Egner et al., 2010; Meyer and Olson, 2011), or after frequent presentation of the same stimulus, in which case the suppression is often described as sensory adaptation (Ul”
Repetition Suppression and Expectation Suppression Are Dissociable in Time in Early Auditory Evoked Fields (2012, background): “…activity in area V1, in connection with expected, simple visual stimuli (den Ouden et al., 2008; Alink et al., 2010; Kok et al., 2011); in the fusiform face area, in connection with faces; and in the parahippocampal place area, in connection with houses (Den Ouden et al., 2010; Egner et al., 2010).”
Concurrent Repetition Enhancement and Suppression Responses in Extrastriate Visual Cortex (2012, background): “Recent computational modeling estimated that prediction error responses outweigh prediction signals by a factor of 2:1 (Egner et al. 2010).”