A continuous semantic space describes the representation of thousands of object and action categories across the human brain

By Alexander G. Huth, Shinji Nishimoto, An T. Vu, Jack L. Gallant. Neuron, 2012

908 citations recorded by the source corpus.

Summary Humans can see and name thousands of distinct object and action categories, so it is unlikely that each category is represented in a distinct brain area. A more efficient scheme would be to represent categories as locations in a continuous semantic space mapped smoothly across the cortical surface. To search for such a space, we used functional magnetic resonance imaging (fMRI) to measure human brain activity evoked by natural movies. We then used voxel-wise models to examine the cortical representation of 1705 object and action categories. The first few dimensions of the underlying semantic space were recovered from the fit models by principal components analysis. Projection of the recovered semantic space onto cortical flat maps shows that semantic selectivity is organized into smooth gradients that cover much of visual and non-visual cortex. Furthermore, both the recovered semantic space and the cortical organization of the space are shared across different individuals.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/f1e5952e2dd3562556693598e04a039712b747d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Attention During Natural Vision Warps Semantic Representation Across the Human Brain (2013, positive): “Complementary tuning-shift analyses were performed in individual ROIs.”
Natural speech reveals the semantic maps that tile human cerebral cortex (2016, positive): “A practical alternative is to project the models into a low-dimensional subspace that retains as much information as possible about the semantic tuning of the voxels 10,14 .”
THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images (2019, background): “Finally, another approach for the selection of object concepts in category selection is to label objects based on their natural appearance in photographs or movies and use these labels in later research (Huth et al., 2012; Russell, Torralba, Murphy, & Freeman, 2008).”
Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision (2016, methodology): “…shown in Figure 3 a , the encoding models could predict cortical responses with reasonably high accuracies for nearly the entire visual cortex, much beyond the spatial extent predictable with low-level visual features (Nishimoto et al. 2011) or high-level semantic features (Huth et al. 2012) alone.”
How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge (2016, methodology): “Equally, if conceptual knowledge relies on specific sensory-based features, would there be a level at which sensory information ‘advances’ towards a more abstract representation?”
Neural Decoding of Visual Imagery During Sleep (2013, methodology): “From the collected reports, words describing visual objects or scenes were manually extracted and mapped to WordNet, a lexical database in which semantically similar words are grouped as “synsets” in a hierarchical structure (17, 18) (Fig.”
A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives. (2015, background): “…(Brouwer and Heeger, 2009), facial identity (Gratton et al., 2013), attention (Sprague and Serences, 2013), working memory (Sprague et al., 2014), numerosity (Harvey et al., 2013), semantics (Huth et al., 2012), and even to other modalities (Schönwiesner and Zatorre, 2009; Thomas et al., 2015).”
The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex (2014, background): “The largescale topography of object representation is reproducible across subjects [33].”
Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses (2014, background): “…work analyzing fMRI from human subjects watching a series of short videos where a large set of objects were identified and annotated with semantic features that were then mapped to brain locations [16], though that work was restricted to semantic features and did not include language stimuli.”
Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation (2014, background): “The IT representation appears to be both visual and semantic Several studies suggested that the IT representation is not purely visual but also semantic [7,14,22,81].”
From brain maps to cognitive ontologies: informatics and the search for mental structure (2015, neutral): “More recent studies have used similar approaches to reconstruct movie clips from visual cortex activation (Nishimoto et al. 2011) or apply semantic labels to thousands of objects and actions (Huth et al. 2012).”