Reverse replay of hippocampal place cells is uniquely modulated by changing reward

By R. E. Ambrose, Brad E. Pfeiffer, David J. Foster. 2016

253 citations recorded by the source corpus.

Summary Hippocampal replays are episodes of sequential place cell activity during sharp wave ripple oscillations (SWRs). Conflicting hypotheses implicate awake replay in learning from rewards, and in memory retrieval for decision-making. Further, awake replays can be forwards, in the same order as experienced, or reverse, in the opposite order. However, while the presence or absence of reward has been reported to modulate SWR rate, the effect of reward changes on replay, and on replay direction in particular, have not been examined. Here we report divergence in the response of forwards and reverse replays to changing reward. While both classes of replays were observed at reward locations, only reverse replays increased their rate at increased reward, or decreased their rate at decreased reward, while forward replays were unchanged. These data demonstrate a unique relationship between reverse replay and reward processing, and point to a functional distinction between different directions of replay.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “The temporal order of replay can be reversed when it is associated with a reward, and the strength of this reversal is correlated with the reward magnitude (Ambrose, Pfeiffer, & Foster, 2016).”
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks (2017, background): “…it has been observed that these replay events are modulated by the reward an animal does or does not receive at the end of a task in the sense that they are more pronounced in the presence of a reward signal and less pronounced or absent in the absence of a reward signal (Ambrose et al., 2016).”
Human Replay Spontaneously Reorganizes Experience (2019, background): “Three features of the current study bring these measurements closer to rodent hippocampal replay: the sequences are present during rest (Foster and Wilson, 2006), they reverse direction after rewards (Ambrose et al., 2016), and they coincide with an increase in source-localized hippocampal power in ”
A dedicated population for reward coding in the hippocampus (2018, unknown): “Other observations have suggested that the computation proceeds in reverse, from the goal to the current location (Ambrose et al., 2016), meaning reward-associated cells could provide a seed for this chain of activations.”
The content of hippocampal “replay” (2018, background): “…indication that awake forward replay may serve to construct predictions regarding future actions came from observations that this form of memory expression tends to arise immediately prior to movement (Ambrose et al., 2016; Diba & Buzs aki, 2007), suggestive of a planning or preparatory mechanism.”
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding (2018, background): “…recordings in rats indicate that brief sequences of prior experience are replayed both in the awake resting state and during sleep, both of which conditions are linked to memory consolidation and learning (Foster & Wilson, 2006; Davidson et al., 2009; Gupta et al., 2010; Ambrose et al., 2016).”
Hippocampal replays under the scrutiny of reinforcement learning models. (2018, background): “Nevertheless, as noted by Lin (1992), replaying trajectories backward, starting from the rewarding location, can sometimes speed up learning; this is in line with the initial interpretation of backward replays (Foster and Wilson 2006) and with the recent observations of Ambrose et al. (2016).”
Learning Structures: Predictive Representations, Replay, and Generalization (2020, background): “These replayed trajectories are not merely memories of recent experience: they are task-related prospective trajectories to remembered goals [50] that are replayed backward when they encounter prediction errors [49].”
Offline replay supports planning in human reinforcement learning (2018, background): “For instance, in more elaborate models, overall variability in outcomes can be decomposed into contributions of volatility and stochasticity, and PE can be scaled relative to these (Ambrose et al., 2016b).”