Reconstructing Speech from Human Auditory Cortex

By Brian N. Pasley, Stephen V. David, Nima Mesgarani, Adeen Flinker, Shihab A. Shamma, Nathan E. Crone, Robert T. Knight, Edward F. Chang. PLoS Biology, 2012

600 citations recorded by the source corpus.

Direct brain recordings from neurosurgical patients listening to speech reveal that the acoustic speech signals can be reconstructed from neural activity in auditory cortex.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/94eb782c85710ce987c9a5183be66872669be8ac

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Emergence of neural encoding of auditory objects while listening to competing speakers (2012, methodology): “These hypothesized, object-specific neural representations are investigated and revealed, using single-trial neural recordings and an advanced neural decoding method that parallels state-of-theart analysis methods used in functional MRI (fMRI) (23) and intracranial recording (24, 25).”
Distinct contributions of low- and high-frequency neural oscillations to speech comprehension (2017, background): “First, the content of heard and attended speech signals can be decoded from high-frequency power (Mesgarani & Chang, 2012; Pasley et al., 2012; Zion Golumbic et al., 2013).”
Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a ‘Cocktail Party’ (2013, background): “This selectivity itself seems to sharpen as a sentence unfolds.”
The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli (2016, methodology): “…inclusion of all response channels in the backward model is advantageous because the reconstruction method gives a low weighting to irrelevant channels whilst allowing the model to capture additional variance using channels potentially excluded by feature selection approaches (Pasley et al., 2012).”
Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. (2015, methodology): “…studies have used recorded cortical population data to estimate the input stimulus using a mapping approach in the reverse direction (i.e., from the neural data back to the stimulus) (Rieke et al. 1995; Stanley et al. 1999; Mesgarani et al. 2009; Pasley et al. 2012; Zion Golumbic et al. 2013).”
Speech synthesis from neural decoding of spoken sentences (2019, background): “Meanwhile, decoding of auditory cortex responses has been more successful for speech sounds 30,31 in part because of the direct relationship between the auditory encoding of spectrotemporal information and the reconstructed spectrogram.”
Neural Cross-Frequency Coupling: Connecting Architectures, Mechanisms, and Functions. (2015, unknown): “While slow oscillations in sensory areas re ﬂ ect the rhythm of the sensory signal or of sensory sampling, fast gamma oscillations underpin ﬁ ne-grained sensory processing in a bottom-up fashion [28,39,74,83].”
Semantic reconstruction of continuous language from non-invasive brain recordings (2022, methodology): “In contrast, our decoder represents language using semantic features and primarily uses data from regions that encode semantic representations 5 during language perception 2 .”
Decoding neural representational spaces using multivariate pattern analysis. (2014, background): “A study of brain activity that closely resembled the original words (Pasley et al. 2012).”
Neural processing of natural sounds (2014, background): “Second, advances in both invasive [133,134] and non-invasive [135] neurophysiological recordings in humans will further allow researchers to make links between animal work and human work.”
A Minimally Invasive 64-Channel Wireless μECoG Implant (2015, background): “I N ORDER to realize the vision of fully autonomous brain–machine interface (BMI) systems, neural implant devices must not only be effective in their function, but should also meet clinical constraints such as ease of implantation, longevity, safety, and small size.”