Decoding and reconstructing color from responses in human visual cortex

By G. Brouwer, David J. Heeger. Journal of Neuroscience, 2009

553 citations recorded by the source corpus.

How is color represented by spatially distributed patterns of activity in visual cortex? Functional magnetic resonance imaging responses to several stimulus colors were analyzed with multivariate techniques: conventional pattern classification, a forward model of idealized color tuning, and principal component analysis (PCA). Stimulus color was accurately decoded from activity in V1, V2, V3, V4, and VO1 but not LO1, LO2, V3A/B, or MT+. The conventional classifier and forward model yielded similar accuracies, but the forward model (unlike the classifier) also reliably reconstructed novel stimulus colors not used to train (specify parameters of) the model. The mean responses, averaged across voxels in each visual area, were not reliably distinguishable for the different stimulus colors. Hence, each stimulus color was associated with a unique spatially distributed pattern of activity, presumably reflecting the color selectivity of cortical neurons. Using PCA, a color space was derived from the covariation, across voxels, in the responses to different colors. In V4 and VO1, the first two principal component scores (main source of variation) of the responses revealed a progression through perceptual color space, with perceptually similar colors evoking the most similar responses. This was not the case for any of the other visual cortical areas, including V1, although decoding was most accurate in V1. This dissociation implies a transformation from the color representation in V1 to reflect perceptual color space in V4 and VO1.

Subfields: Atomic and Molecular Physics, and Optics, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/7777f6e5a13aaf197b89964723d2ec8eb87cc200

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Cognitive Neuroscience

Citations in the corpus

A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives. (2015, methodology): “However, Brouwer and Heeger (2009) showed that this need not be the case.”
Imaging retinotopic maps in the human brain (2010, background): “…to understanding the function of cortical areas compares the similarity of BOLD responses within a region of interest to the similarity of judgments made by the subject (Brouwer & Heeger , 2009; Haushofer, Livingstone, & Kanwisher, 2008; Weber, Thompson-Schill, Osherson, Haxby, & Parsons, 2009).”
Encoding and decoding in fMRI (2010, background): “An incomplete tally of some current work includes studies of vision (Brouwer and Heeger, 2009; Carlson et al., 2003; Cox and Savoy, 2003; Eger et al., 2008; Haxby et al., 2001; MacEvoy and Epstein, 2009; Peelen et al., 2009; Walther et al., 2009),”
Reconstructing visual experiences from brain activity evoked by natural movies (2011, unknown): “(Note that when isoluminant, monochromatic stimuli are used, color can be reconstructed from evokedBOLD signals [39].”
Foundations of Vision (2018, background): “Such approaches have been used to demonstrate compelling links between color perception and cortical responses in area V4 (Brouwer & Heeger, 2009) and to distinguish among hundreds of natural scene images (Kay et al., 2008).”