Selective cortical representation of attended speaker in multi-talker speech perception

By Nima Mesgarani, Edward F. Chang. Nature, 2012

1,013 citations recorded by the source corpus.

Humans possess a remarkable ability to attend to a single speaker’s voice in a multi-talker background. How the auditory system manages to extract intelligible speech under such acoustically complex and adverse listening conditions is not known, and, indeed, it is not clear how attended speech is internally represented. Here, using multi-electrode surface recordings from the cortex of subjects engaged in a listening task with two simultaneous speakers, we demonstrate that population responses in non-primary human auditory cortex encode critical features of attended speech: speech spectrograms reconstructed based on cortical responses to the mixture of speakers reveal the salient spectral and temporal features of the attended speaker, as if subjects were listening to that speaker alone. A simple classifier trained solely on examples of single speakers can decode both attended words and speaker identity. We find that task performance is well predicted by a rapid increase in attention-modulated neural selectivity across both single-electrode and population-level cortical responses. These findings demonstrate that the cortical representation of speech does not merely reflect the external acoustic environment, but instead gives rise to the perceptual aspects relevant for the listener’s intended goal.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/9f992b8e07689b8ae3da1a761472daeb82c544c9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Emergence of neural encoding of auditory objects while listening to competing speakers (2012, methodology): “Taken together, these results support the idea that posterior auditory cortex plays a major role in the generation of auditory objects (42, 43) and the selection of information based on the listener’s interest (25, 44).”
Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a ‘Cocktail Party’ (2013, background): “…of Physicians and Surgeons, New York, NY, USA 4Cognitive Neuroscience and Schizophrenia Program, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY 10962, USA 5Department of Electrical and Computer Engineering 6Department of Biology University of Maryland, College Park, College…”
The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli (2016, methodology): “…SI techniques (that ultimately yield the same solution) have also been used to investigate auditory scene analysis (Ding and Simon, 2012a; Mesgarani and Chang, 2012; Zion-Golumbic et al., 2013; O’Sullivan et al., 2015), speech-in-noise (Ding and Simon, 2013; Ding et al., 2014), overt and…”
Speaker-Independent Speech Separation With Deep Attractor Network (2017, background): “Humans are particularly adept at this task, even in the absence of any spatial separation between speakers [3]–[5].”
Distinct contributions of low- and high-frequency neural oscillations to speech comprehension (2017, background): “First, the content of heard and attended speech signals can be decoded from high-frequency power (Mesgarani & Chang, 2012; Pasley et al., 2012; Zion Golumbic et al., 2013).”
The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances (2013, background): “This WMC-based top-down influence on speech-relevant attention processes may be part of the explanation as to why attending to a speaker in a multitalker situation gives rise to dedicated neural representations (Mesgarani and Chang, 2012).”
Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. (2015, background): “…Trinity College Dublin, Dublin 2, Ireland, 2Department of Psychology, Centre for Neuroscience in Education, University of Cambridge, Cambridge, UK, 3Department of Neurological Surgery, 4Department of Physiology, UCSF Center for Integrative Neuroscience, University of California, San…”
Bridging verbal coordination and neural dynamics (2025, unknown): “In these studies, modulations corresponded to increases in activity in response to targets compared to non-targets, which have been interpreted in terms of selective attention or a bias toward behaviourally relevant stimuli (Petkov et al., 2004 ; Mesgarani & Chang, 2012).”
Intracranial Mapping of Response Latencies and Task Effects for Spoken Syllable Processing in the Human Brain (2024, unknown): “…brain region by finding the region label of the closest voxel. sEEG pre-processing In line with pre-processing procedures used in prior literature (Mesgarani and Chang, 2012; Mesgarani et al., 2014; Hamilton et al., 2020), the raw sEEG data were re-referenced to the average signal across channels.”