Phonetic Feature Encoding in Human Superior Temporal Gyrus

By Nima Mesgarani, Connie Cheung, Keith Johnson, Edward F. Chang. Science, 2014

954 citations recorded by the source corpus.

Deciphering Language Consonants and vowels represent basic building blocks of human language. How their characteristics are extracted from acoustic speech input is not well understood. Directly recording from the superior temporal gyrus of patients as part of their clinical evaluation for epilepsy surgery, Mesgarani et al. (p. 1006, published online 30 January; see the Perspective by Grodzinsky and Nelken) investigated neural responses while the subjects listened to continuous speech. The findings reveal how both vowels and consonants of different phonetic categories are encoded. The human auditory cortex encodes what speech sounds like. [Also see Perspective by Grodzinsky and Nelken] During speech perception, linguistic elements such as consonants and vowels are extracted from a complex acoustic speech signal. The superior temporal gyrus (STG) participates in high-order auditory processing of speech, but how it encodes phonetic information is poorly understood. We used high-density direct cortical surface recordings in humans while they listened to natural, continuous speech to reveal the STG representation of the entire English phonetic inventory. At single electrodes, we found response selectivity to distinct phonetic features. Encoding of acoustic properties was mediated by a distributed population response. Phonetic features could be directly related to tuning for spectrotemporal acoustic cues, some of which were encoded in a nonlinear fashion or by integration of multiple cues. These findings demonstrate the acoustic-phonetic representation of speech in human STG.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology, Experimental and Cognitive Psychology

View Paper: https://www.semanticscholar.org/paper/11c02a1b64b833115522f781e219b1243498b551

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

In Experimental and Cognitive Psychology

Citations in the corpus

Illiterate to literate: behavioural and cerebral changes induced by reading acquisition (2015, positive): “Together with surrounding superior temporal and supramarginal cortices, the PT houses a neuronal representation of the consonants and vowels of spoken language 96,97 .”
Toward Human Parity in Conversational Speech Recognition (2017, methodology): “Inspired by the human auditory cortex [57], where neighboring neurons tend to activate simultaneously, we employ a spatial smoothing technique to improve the accuracy of our”
Exploring how deep neural networks form phonemic categories (2015, background): “In addition, our results closely match the recent findings of speech encoding in the human auditory system [12], where phonetic features were found to similarly organize the cortical representation of speech in higher level auditory areas.”
Speech synthesis from neural decoding of spoken sentences (2019, methodology): “Third, since the success of the decoder depends on the initial electrode placement, we quantified the contribution of several anatomical regions (vSMC, STG, and IFG) that are involved in continuous speech production 25 .”
Discriminative template learning in group-convolutional networks for invariant speech representations (2015, background): “Apart from the sophisticated contextual inference through complex language models, the lower-level neural representation of speech sounds might also be important for speech recognition [5, 6].”
Distinct contributions of low- and high-frequency neural oscillations to speech comprehension (2017, background): “High-frequency activity would thus reflect the read-out of speech processing (Ghitza, 2011; Giraud & Poeppel, 2012; Mesgarani et al., 2014; Poeppel, 2003; Poeppel et al., 2008).”
A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy. (2018, neutral): “Speech-selective responses emerge only in non-primary areas (de Heer et al., 2017; Evans and Davis, 2015; Mesgarani et al., 2014; Okada et al., 2010; Overath et al., 2015) and cannot be accounted for by modulation features standardly used to model primary areas (Norman-Haignere et al., 2015;…”
Mouth and Voice: A Relationship between Visual and Auditory Preference in the Human Superior Temporal Sulcus (2017, background): “…neurons in the pSTS to represent both the visual features of mouth movements (Bernstein and Liebenthal, 2014; Irwin et al., 2015) and the auditory features of speech (Mesgarani et al., 2014; Arsenault and Buchsbaum, 2015; Correia et al., 2015; Norman-Haignere et al., 2015; Brewer and Barton, 2016).”
Neural population coding: combining insights from microscopic and mass signals (2015, background): “…recent neuroimaging studies extracted features of population activity inﬂuencing the variability of single-trial percepts [82–84], and demonstrated that visual object categories [85] or ﬁne details of auditory signals [86–90] can be recovered using stimulus reconstruction or decoding methods.”
Neurophysiological dynamics of phrase-structure building during sentence processing (2017, background): “The intracranial approach to language processing has led to important advances in phonetics (19) and single-word processing (20).”