Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses

By Leila Wehbe, Brian Murphy, Partha P. Talukdar, Alona Fyshe, Aaditya Ramdas, Tom Michael Mitchell. PLoS ONE, 2014

312 citations recorded by the source corpus.

Story understanding involves many perceptual and cognitive subprocesses, from perceiving individual words, to parsing sentences, to understanding the relationships among the story characters. We present an integrated computational model of reading that incorporates these and additional subprocesses, simultaneously discovering their fMRI signatures. Our model predicts the fMRI activity associated with reading arbitrary text passages, well enough to distinguish which of two story segments is being read with 74% accuracy. This approach is the first to simultaneously track diverse reading subprocesses during complex story processing and predict the detailed neural representation of diverse story features, ranging from visual word properties to the mention of different story characters and different actions they perform. We construct brain representation maps that replicate many results from a wide range of classical studies that focus each on one aspect of language processing and offer new insights on which type of information is processed by different areas involved in language processing. Additionally, this approach is promising for studying individual differences: it can be used to create single subject maps that may potentially be used to measure reading comprehension and diagnose reading disorders.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology

View Paper: https://www.semanticscholar.org/paper/22cebf22de747bd68beb8820406c7c52979ee9ed

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

Citations in the corpus

Natural speech reveals the semantic maps that tile human cerebral cortex (2016, unknown): “Data-driven approaches are commonplace in studies of human neuroanatomy 31 and resting state networks 26,32 , but are only beginning to be used in functional imaging 14,15 .”
Additive Approximations in High Dimensional Nonparametric Regression via the SALSA (2016, methodology): “Brain: (From Wehbe et al. (2014)), Predictor: Story feature at a given time step Features: Other attributes Some experimental details: GP is the Bayesian interpretation of KRR.”
Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks (2015, methodology): “As an example, attention-based models can be used in neuroscience to temporally and spatially map between the neuronal activities and a sequence of stimuli [75].”
Blackbox Meets Blackbox: Representational Similarity & Stability Analysis of Neural Language Models and Brains (2019, unknown): “models, or using NLP models to build predictive models of the human brain, or both (Murphy et al., 2012; Wehbe et al., 2014a; Ruan et al., 2016; Søgaard, 2016; Xu et al., 2016; Fyshe et al., 2014; Bingel et al., 2016; Bulat et al., 2017; Abnar et al., 2018; Pereira et al., 2018; Huth et al., 2016).”
Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) (2019, methodology): “The same chapter was shown by Wehbe et al. (2014a) to 3 subjects in MEG with the same rate of 0.5 seconds per word.”
Encoding of phonology in a recurrent neural model of grounded speech (2017, methodology): “More recently, naturalistic data has been used and patterns in the brain have been correlated with patterns in the input (e.g. Wehbe et al., 2014; Khalighinejad et al., 2017).”
Estimating Accuracy from Unlabeled Data (2014, background): “Additional details can be found in [Wehbe et al., 2014].”