Hippocampal place cells construct reward related sequences through unexplored space

By H. F. Ólafsdóttir, C. Barry, Aman B. Saleem, Demis Hassabis, Hugo J. Spiers. 2015

277 citations recorded by the source corpus.

Dominant theories of hippocampal function propose that place cell representations are formed during an animal's first encounter with a novel environment and are subsequently replayed during off-line states to support consolidation and future behaviour. Here we report that viewing the delivery of food to an unvisited portion of an environment leads to off-line pre-activation of place cells sequences corresponding to that space. Such ‘preplay’ was not observed for an unrewarded but otherwise similar portion of the environment. These results suggest that a hippocampal representation of a visible, yet unexplored environment can be formed if the environment is of motivational relevance to the animal. We hypothesise such goal-biased preplay may support preparation for future experiences in novel environments. DOI: http://dx.doi.org/10.7554/eLife.06063.001

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Neuroscience

Citations in the corpus

Prioritized Experience Replay (2015, background): “Sequences associated with rewards appear to be replayed more frequently (Atherton et al., 2015; ´Olafsd´ottir et al., 2015; Foster & Wilson, 2006).”
Selective Experience Replay for Lifelong Learning (2018, background): “Reward Neuroscience research suggests replay occurs more often for rewarding events (Atherton, Dupret, and Mel-lor 2015; ´Olafsd´ottir et al. 2015).”
Navigating cognition: Spatial codes for human thinking (2018, background): “Similarly, hippocampal place cells constructed trajectories through previously nontraversed space after rats observed the placement of a reward on one T-maze arm while confined to the stem of the maze (129).”
How Memory Replay in Sleep Boosts Creative Problem-Solving (2018, background): “Fascinatingly, recent work has demonstrated that novel (as yet untraveled) trajectories which would allow an animal to reach food are sometimes ‘preplayed’ by the hippocampus, potentially showing the neural basis of problem solving [78,79].”
Integrating memories to guide decisions (2015, background): “Such prioritization has been the product of some work in computer science [54] though — apart from a finding that place cell sequences favor unexplored locations that contain reward over those without it [39] — there is almost no neuroscientific data addressing this question.”
The Role of Hippocampal Replay in Memory and Planning (2018, background): “This does appear to occur: in fact, replayed sequences have been reported to combine elements of an environment that were experienced separately and even to represent paths through a section of the environment the animal had seen but not experienced [48,49].”
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior. (2018, background): “The green ‘‘G’’ indicates the goal location. in neural processes that relate to a broader view of a cognitive map, such as generalization, inference, imagination, social cognition, and memory (Hassabis et al., 2007; van der Meer et al., 2012; Ólafsdóttir et al., 2015; Tavares et al., 2015).”
On the link between conscious function and general intelligence in humans and machines (2022, background): “We make the further distinction between mental time travel and experience preplay, which consists of the simulation of future experiences within a limited environmental and behavioral context (Pezzulo et al., 2014; Ólafsdóttir et al., 2015).”
Offline replay supports planning in human reinforcement learning (2018, background): “Indeed, this hypothesis fits closely with a substantial rodent literature revealing that hippocampal replay of previously experienced spatial trajectories is observed during both awake behavior and sleep (Takahashi, 2015; Ólafsdóttir et al., 2015).”
Learning Structures: Predictive Representations, Replay, and Generalization (2020, background): “Human and animal neural evidence supports a role for wake and sleep memory replay in generalization and community detection [59,60], inference [52,61] especially of unseen relations among states, problem solving, and memory consolidation [62,63].”
The successor representation in human reinforcement learning (2016, background): “Evidence from rodents and human studies showing that offline replay of sequences during rest and sleep enhances memory consolidation 31 and learning new trajectories 32,33 .”
Hippocampal Reactivation of Random Trajectories Resembling Brownian Diffusion. (2019, background): “Indeed, it has been shown that firing patterns resembling that of novel paths can also be seen in sleep prior to the animal experiencing that path, termed preplay(Dragoi and Tonegawa, 2011; Grosmark and Buzsáki, 2016; Ólafsdóttir et al., 2015).”
Neuron Clustering for Mitigating Catastrophic Forgetting in Supervised and Reinforcement Learning (2015, background): “It has been hypothesized that this replay may be connected to learning reward driven behavior [47].”
Neuroscience-Inspired Artificial Intelligence. (2017, unknown): “…ripples of neural activity in the rat hippocampus resemble those observed during subsequent navigation of the available trajectories (‘‘preplay’’), as if the animal were ‘‘imag-ining’’ each possible alternative (Johnson and Redish, 2007; O´lafsdo´ttir et al., 2015; Pfeiffer and Foster, 2013).”
Using Grid Cells for Navigation (2015, unknown): “Interestingly, recent data indicates that place cell ripple related preplay can include novel routes (O´lafsdo´ttir et al., 2015)—a key property of vector navigation.”