Sequential Replay of Non-spatial Task States in the Human Hippocampus

By Nicolas W. Schuck, Yael Niv. 2018

237 citations recorded by the source corpus.

Replay of activity in the human brain Electrophysiological recordings in rats and mice have shown that specific hippocampal neuronal activity patterns are sequentially reactivated during rest periods or sleep. Does the human hippocampus also replay activity sequences, even in a nonspatial task, such as, for example, decision-making? Schuck and Niv studied functional magnetic resonance imaging signals in subjects after they had learned a decision-making task. While people rested, the replay of activity patterns in the hippocampus reflected the order of previous task-state sequences. Thus, sequential hippocampal reactivation might participate in decision-making in humans. Science, this issue p. eaaw5181 Reactivations of human hippocampal functional MRI signals reflect the order of previous task state sequences. INTRODUCTION The hippocampus plays an important role in memory and spatial navigation. When rodents navigate a spatial maze, hippocampal neurons called place cells show spatially selective response fields, activating only during visitation of particular places. In this way, during navigation, place cells activate sequentially, reflecting traveled paths. During sleep and wakeful rest, the same sequences of place cells are reactivated from memory, or replayed, although the animal is stationary. Replayed sequences are temporally compressed, occurring on the order of 100 ms, and have been linked to an offline sampling process that is important for memory consolidation. Advances in reinforcement learning, an area of machine learning, suggest that offline experience replay may also serve computational functions underlying nonspatial learning and decision-making. RATIONALE The study of hippocampal replay in the human brain is challenging because noninvasive neuroimaging techniques have either relatively low spatial or temporal resolution. Nevertheless, we reasoned that fast neuronal replay events may be detectable in blood oxygen level–dependent (BOLD) signals recorded with functional magnetic resonance imaging (fMRI) because the prolonged BOLD response translates short neural events into long-lasting signals. By applying multivariate decoding techniques that can disentangle subtle and spatially overlapping activity patterns, it may therefore be possible to detect fast replay events as ordered activation of sequential fMRI patterns. Studying hippocampal replay in humans allows investigation of abstract, nonspatial tasks to determine the extent to which the hippocampus is important for sequential memory and decision-making more broadly. RESULTS We measured fMRI BOLD signals while human participants performed a nonspatial decision-making task and while participants rested before and after completing the task. A support vector machine classifier was then trained on labeled task data from the hippocampus and applied to multivariate time courses acquired during the rest sessions. We found that sequences of patterns decoded from the hippocampus as participants rested after task performance reflected the order of previous experiences, with consecutively decoded task states being “nearby” in the abstract task-state diagram. This ordering of successive fMRI patterns reflected sequences of task states rather than simpler sequences of attentional or sensory experiences. Moreover, the extent of this hippocampal offline replay was related to the integrity of on-task representation of task states in the orbitofrontal cortex, an area previously shown to be important for representing the current task state during decision-making. On-task encoding of task states in the orbitofrontal cortex was further related to behavioral performance, suggesting a role for hippocampal replay in training task-relevant representations in the orbitofrontal cortex. Experimental control conditions and permutation analyses supported these results, and simulations showed that our proposed statistical analyses are, in principle, sensitive to sequential neural events occurring on the order of 100 ms—the time resolution relevant for replay events. CONCLUSION Our results support the importance of sequential reactivation in the human hippocampus for nonspatial decision-making and establish the feasibility of investigating such rapid signals with fMRI, despite substantial limitations in temporal resolution. Decoding sequential replay with fMRI. (Top) Participants made age judgments of either faces or houses for a sequence of overlaid face-house images while brain activity was recorded with fMRI. Task rules required keeping in mind the age and judged category of the current and previous trial, called task states. (Bottom) The task states followed a predefined sequential structure. A pattern classifier was trained to classify the 16 task states from on-task hippocampal fMRI data (illustrated with orange patterns). (Middle) The classifier was then applied to fMRI data recorded during wakeful rest in the same participants to decode potentially replayed sequences of task states (lines connecting patterns in top and middle). Sequences of decoded task states were related to the sequential structure of the task (bottom) by counting how many steps separated every two consecutive decoded states in the true task structure (green circles; red circles indicate states that were “skipped” in the decoding). Skips omitting fewer task states between successive decoded states were more frequent in the resting data than in control data, indicating sequential replay of nonspatial task states in the hippocampus during wakeful rest. Sequential neural activity patterns related to spatial experiences are “replayed” in the hippocampus of rodents during rest. We investigated whether replay of nonspatial sequences can be detected noninvasively in the human hippocampus. Participants underwent functional magnetic resonance imaging (fMRI) while resting after performing a decision-making task with sequential structure. Hippocampal fMRI patterns recorded at rest reflected sequentiality of previously experienced task states, with consecutive patterns corresponding to nearby states. Hippocampal sequentiality correlated with the fidelity of task representations recorded in the orbitofrontal cortex during decision-making, which were themselves related to better task performance. Our findings suggest that hippocampal replay may be important for building representations of complex, abstract tasks elsewhere in the brain and establish feasibility of investigating fast replay signals with fMRI.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Schuck and Niv (2019) also suggest that hippocampal replay may be important for building representations of complex, abstract tasks elsewhere in the brain.”
Lifelong Generative Modeling (2017, background): “While strongly motivated by biological rehearsal processes [119, 53, 58, 110], these generative replay strategies fail to efficiently use previous learning and simply re-learn each new joint task from scratch.”
Navigating cognition: Spatial codes for human thinking (2018, background): “Initial evidence suggests that sequential hippocampal activity can be observed using fMRI (126).”
If deep learning is the answer, what is the question? (2020, unknown): “Nevertheless, inventive new ways of using these tools are being developed, allowing researchers to probe replay 86–88 , changes in excitatory–inhibitory balance 89,90 and hexagonal (grid) coding 91,92 in human brain signals.”
Automatic Recall Machines: Internal Replay, Continual Learning and the Brain (2020, background): “This forms a parallel with the biological brain, where increased hippocampal replay leads to increased task performance or reward [Schuck and Niv, 2019] and vice versa [Mattar and Daw, 2018].”