Discovering event structure in continuous narrative perception and memory

By Christopher A. Baldassano, Janice Chen, Asieh Zadbood, Jonathan W. Pillow, Uri Hasson, Kenneth A. Norman. 2016

817 citations recorded by the source corpus.

Summary During realistic, continuous perception, humans automatically segment experiences into discrete events. Using a novel model of neural event dynamics, we investigate how cortical structures generate event representations during continuous narratives, and how these events are stored and retrieved from long-term memory. Our data-driven approach enables identification of event boundaries and event correspondences across datasets without human-generated stimulus annotations, and reveals that different regions segment narratives at different timescales. We also provide the first direct evidence that narrative event boundaries in high-order areas (overlapping the default mode network) trigger encoding processes in the hippocampus, and that this encoding activity predicts pattern reinstatement during recall. Finally, we demonstrate that these areas represent abstract, multimodal situation models, and show anticipatory event reinstatement as subjects listen to a familiar narrative. Our results provide strong evidence that brain activity is naturally structured into semantically meaningful events, which are stored in and retrieved from long-term memory.

Subfields: Cognitive Neuroscience, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Cognitive Science & Psychology

Citations in the corpus

Are episodic memories special? On the sameness of remembered and imagined event simulation (2018, background): “The second perspective comes from two related theories of how the continuous stream of experience is processed in the present moment: event segmentation theory (Zacks et al. 2007) and the hierarchical process memory framework (Hasson et al. 2015; Chen et al. 2016; Baldassano et al. 2017).”
20 years of the default mode network: a review and synthesis (2023, background): “94,95 The DMN has also been shown to represent event boundaries in both external narratives 96,97 and internally generated boundaries of mental context during un-guided narrative recall, 98 with neural patterns that generalize across both internally and externally driven boundaries being most…”
Mapping the human brain’s cortical-subcortical functional network organization (2018, neutral): “Additionally, PMM might be important for identifying and representing event structure in narratives, given that PMM regions were recently shown to represent long narrative structures during movie watching (Baldassano et al., 2017).”
Large-Scale Gradients in Human Cortical Organization. (2018, neutral): “(D) The length of events that are represented in a given area, here extracted from movie-watching data, varies from short events in sensory areas to long events in transmodal regions (only patterns with high between-subject consistency are shown, for example, somatosensory regions did not respond co”
Compositional Imitation Learning: Explaining and executing one task at a time (2018, background): “Humans are adept at inferring event structure by hierarchically segmenting continuous sensory experience [1, 2, 3], which may support building efficient event representations in episodic memory [4] and constructing abstract plans [5].”
CompILE: Compositional Imitation Learning and Execution (2018, background): “Humans are adept at inferring event structure by hierarchically segmenting continuous sensory experience (Zacks et al., 2001; Baldassano et al., 2017; Radvansky & Zacks, 2017), which may support building efficient event representations in episodic memory (Ezzyat & Davachi, 2011) and constructing…”
Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models (2020, background): “…that represent scenes extending over tens of seconds (e.g., a detective searching a suspect ’ s apartment for evidence) are largely the same, whether the information comes from watching a movie, hearing or reading a narrative description, or recalling the movie after having seen it (54, 55).”
Switching between internal and external modes: A multiscale learning principle (2017, background): “…occurs on the timescale of seconds, then information about the prior state can be stored in circuit activation (e.g., a slowly changing distributed pattern; Baldassano et al., 2016) and differences may be computed continuously in all phases (Hasselmo et al., 2002; Norman et al., 2006).”
Neural reactivation in parietal cortex enhances memory for episodically linked information (2018, background): “In contrast, evidence reviewed above suggests that other parietal regions in the PM network may represent event content in a manner that generalizes across sensory modalities (38).”
Extending Machine Language Models toward Human-Level Language Understanding (2019, background): “, a detective searching a suspect’s apartment for evidence) are largely the same, whether the information comes from watching a movie, hearing or reading a narrative description, or recalling the movie after having seen it (52, 53).”
Navigating cognition: Spatial codes for human thinking (2018, background): “Multiscale event representations, interacting with mnemonic processing in the hippocampus, have been dissociated along the cortical hierarchy in humans (95).”
Aperiodic neural timescales in prefrontal cortex dilate with increased task abstraction (2025, unknown): “This literature broadly suggests that continuous low-level sensory inputs received by fast-timescale brain regions are progressively integrated into abstract and context-sensitive representations by higher-order, longer-timescale brain regions (18,75–85).”
The temporal dynamics of how the brain structures natural scenes. (2023, unknown): “…related to the event segmentation processes in several brain regions, for instance, the lateral prefrontal cortex and the anterior cingulate cortex (Baldassano et al., 2017; Hasson, Nir, Levy, Fuhrmann, & Malach, 2004; Kurby & Zacks, 2008; Magliano & Zacks, 2011; Speer, Zacks, & Reynolds, 2007,…”
The left angular gyrus is causally involved in context-dependent integration and associative encoding during narrative reading (2021, methodology): “the AG role observed in previous studies (e.g., Branzi et al., 2020; Baldassano et al., 2017; Humphries et al., 2007) may be causally related to such processes.”