Natural speech reveals the semantic maps that tile human cerebral cortex

By Alexander G. Huth, Wendy A. de Heer, T. Griffiths, Frédéric E. Theunissen, Jack L. Gallant. Nature, 2016

1,473 citations recorded by the source corpus.

The meaning of language is represented in regions of the cerebral cortex collectively known as the ‘semantic system’. However, little of the semantic system has been mapped comprehensively, and the semantic selectivity of most regions is unknown. Here we systematically map semantic selectivity across the cortex using voxel-wise modelling of functional MRI (fMRI) data collected while subjects listened to hours of narrative stories. We show that the semantic system is organized into intricate patterns that seem to be consistent across individuals. We then use a novel generative model to create a detailed semantic atlas. Our results suggest that most areas within the semantic system represent information about specific semantic domains, or groups of related concepts, and our atlas shows which domains are represented in each area. This study demonstrates that data-driven methods—commonplace in studies of human neuroanatomy and functional connectivity—provide a powerful and efficient means for mapping functional representations in the brain.

Subfields: Cognitive Neuroscience, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/ad22b61cb977377cc422b0b39a0bc5c58f1ab1ea

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Molecular Biology

Citations in the corpus

Neurodevelopment of the association cortices: Patterns, mechanisms, and implications for psychopathology (2021, neutral): “…neural encodingcouldbemappedacross theS-A axis during neurodevelopment by evaluating areal dimensionality of stimulus representations, in particular for stimuli that are both diverse and represented across most of the cortex (e.g., natural language and semantic representations; Huth et al., 2016).”
The neural architecture of language: Integrative modeling converges on predictive processing (2020, methodology): “For example, several earlier studies used simple embedding models to decode linguistic meaning from fMRI data (e.g., Wehbe et al., 2014; Huth et al., 2016; Anderson et al., 2017; Pereira et al., 2018) and reported reliable decoding not only within the language network, but also across other parts…”
Large-Scale Gradients in Human Cortical Organization. (2018, neutral): “In support of this view, spatial gradients of functional integration ranging from unimodal to transmodal areas have recently been described in the human cortex [35,46].”
Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision (2016, methodology): “Since deep-learning models are already available for speech recognitions (Hinton et al. 2012) and language processing (Collobert and Weston 2008), decoding of brain measures in response to natural hearing, speech, and language are realistically attainable goals (Huth et al. 2016a).”
Brains and algorithms partially converge in natural language processing (2022, methodology): “This result not only con ﬁ rms the wide spread distribution of meaning in the brain 44,64 , but also reveals its remarkably long-lasting nature.”
Blackbox Meets Blackbox: Representational Similarity & Stability Analysis of Neural Language Models and Brains (2019, methodology): “models, or using NLP models to build predictive models of the human brain, or both (Murphy et al., 2012; Wehbe et al., 2014a; Ruan et al., 2016; Søgaard, 2016; Xu et al., 2016; Fyshe et al., 2014; Bingel et al., 2016; Bulat et al., 2017; Abnar et al., 2018; Pereira et al., 2018; Huth et al., 2016).”
Cognitive computational neuroscience (2018, background): “Representational models are often defined on the basis of descriptions of the stimuli, such as labels provided by human observers.(91,98,98,99) In this scenario, a representational model that explains the brain responses in a given region provides, not a brain-computational account, but at least a d”
Explaining Patterns in Data with Language Models via Interpretable Autoprompting (2022, background): “A recent seminal study [10] explores this question by investigating where different natural-language categories are represented in the human neocortex.”
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) (2019, methodology): “A few researchers instead use naturalistic stimulus such as stories (Brennan et al., 2010; Lerner et al., 2011; Speer et al., 2009; Wehbe et al., 2014b; Huth et al., 2016; Blank and Fedorenko, 2017).”
Precision functional mapping of individual human brains (2017, positive): “…generally shied away from studying individuals,with the notable exception of studies focusedon specific regions of the cortex using functional localizers (Kanwisher, 2017) or high-field imaging (Cheng, 2016) (though see Huth et al., 2016 for an alternative approach to studying individual brains).”
Imaging-based parcellations of the human brain (2018, methodology): “…Kong2019_MSHBM 5 Other PrAGMATiC, based on task fMRI Cerebral cortex 320 parcels Volume (and surface) For visualization only: http://gallantlab. org/huth2016/ 33,149 Brainnetome, based on PDT Cerebral cortex and subcortical structures 246 parcels Volume http://atlas.brainnetome.org/download.…”
Essentials of a Theory of Language Cognition (2019, background): “, 2008] and Natural Speech reveals the Semantic Maps That Tile Human Cerebral Cortex [Huth et al., 2016]).”
Building a Science of Individual Differences from fMRI. (2016, neutral): “Such work is already well underway [199].”
Semantic reconstruction of continuous language from non-invasive brain recordings (2022, methodology): “To compare word sequences to a subject’s brain responses, we used an encoding model 5 that predicts how the subject’s brain responds to natural language.”