Decoding reveals the contents of visual working memory in early visual areas

By S. Harrison, Frank Tong. Nature, 2009

1,299 citations recorded by the source corpus.

Visual working memory provides an essential link between perception and higher cognitive functions, allowing for the active maintenance of information about stimuli no longer in view. Research suggests that sustained activity in higher-order prefrontal, parietal, inferotemporal and lateral occipital areas supports visual maintenance, and may account for the limited capacity of working memory to hold up to 3–4 items. Because higher-order areas lack the visual selectivity of early sensory areas, it has remained unclear how observers can remember specific visual features, such as the precise orientation of a grating, with minimal decay in performance over delays of many seconds. One proposal is that sensory areas serve to maintain fine-tuned feature information, but early visual areas show little to no sustained activity over prolonged delays. Here we show that orientations held in working memory can be decoded from activity patterns in the human visual cortex, even when overall levels of activity are low. Using functional magnetic resonance imaging and pattern classification methods, we found that activity patterns in visual areas V1–V4 could predict which of two oriented gratings was held in memory with mean accuracy levels upwards of 80%, even in participants whose activity fell to baseline levels after a prolonged delay. These orientation-selective activity patterns were sustained throughout the delay period, evident in individual visual areas, and similar to the responses evoked by unattended, task-irrelevant gratings. Our results demonstrate that early visual areas can retain specific information about visual features held in working memory, over periods of many seconds when no physical stimulus is present.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/b02c533c2143cfc22259ef9bd4114411f8f33603

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Varieties of cognitive penetration in visual perception. (2014, neutral): “…influenced by the information content of the visual surround (Smith & Muckli, 2010), by the content of memory even in the absence of a visual stimulus (Harrison & Tong, 2009) and by the content of what is heard or imagined, again in the absence of a visual stimulus (Vetter, Smith, & Muckli, 2014).”
Decoding neural representational spaces using multivariate pattern analysis. (2014, background): “Soon et al. 2008) and the contents of working memory (Harrison & Tong 2009).”
Reconstructing visual experiences from brain activity evoked by natural movies (2011, neutral): “Other studies have decoded subjective mental states, such as the contents of visual workingmemory [42], or whether subjects are attending to one or another orientation or direction [3, 43].”
The Cambridge Centre for Ageing and Neuroscience (Cam-CAN) study protocol: a cross-sectional, lifespan, multidisciplinary examination of healthy cognitive ageing (2014, background): “Neurally, the capacity limit is reflected in posterior parietal cortex [52,53], although VSTM tasks activate a frontal-parietal “multiple demand” network [54], and visual memories can be decoded from occipital cortex [55,56].”
A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes (2014, neutral): “Their results serve as a critical starting point for consolidating the extensive and largely parallel literatures onworkingmemory andmental imagery (Harrison and Tong, 2009; Tong, 2013).”
A taxonomy of external and internal attention. (2011, neutral): “Working memory maintenance directly modulates processing in relevant sensory cortex (Harrison & Tong 2009, Serences et al. 2009), and suppresses irrelevant information (Gazzaley et al. 2005).”
Reading the mind's eye: decoding category information during mental imagery (2009, background): “Finally, very recently Harrison and Tong (2009) showed that orientation information held in working memory could be decoded from fMRI activity in areas V1–V4.”
Report Decoding Sound and Imagery Content in Early Visual Cortex (2014, neutral): “For example, activity patterns in nonstimulated parts of early visual cortex contain content-specific information from the surrounding visual context [31, 32], from objects presented in the periphery [33], and from visual stimuli solely held in working memory rather than being actively perceived [34”
Mental Imagery: Functional Mechanisms and Clinical Applications (2015, neutral): “One involves creating a mental image to compare with the subsequent test stimuli [26–28]; the other strategy involves picking out particular details of a scene or array and encoding them phonologically or verbally, which is then compared with the test stimuli [26,28,29].”
A neural basis for real-world visual search in human occipitotemporal cortex (2011, background): “Other studies have reported top-down activity of visual cortex in paradigms where participants were asked to visually memorize the orientation of a gabor stimulus (17, 18) or its color (18), were expecting a particular object shape (10–12), or were asked to vividly imagine a specific category exempl”
The Distributed Nature of Working Memory. (2017, background): “(1) Local pattern information: it has been repeatedly shown that the orientation of a grating stimulus during a working memory delay can be decoded from patterns of brain activity in primary visual cortex [79].”