Hippocampal replays under the scrutiny of reinforcement learning models.
By Romain D. Cazé, Mehdi Khamassi, Lise Aubin, Benoît Girard. 2018
42 citations recorded by the source corpus.
Multiple in vivo studies have shown that place cells from the hippocampus replay previously experienced trajectories. These replays are commonly considered to mainly reflect memory consolidation processes. Some data, however, have highlighted a functional link between replays and reinforcement learning (RL). This theory, extensively used in machine learning, has introduced efficient algorithms and can explain various behavioral and physiological measures from different brain regions. RL algorithms could constitute a mechanistic description of replays and explain how replays can reduce the number of iterations required to explore the environment during learning. We review the main findings concerning the different hippocampal replay types and the possible associated RL models (either model-based, model-free, or hybrid model types). We conclude by tying these frameworks together. We illustrate the link between data and RL through a series of model simulations. This review, at the frontier between informatics and biology, paves the way for future work on replays.
Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cellular and Molecular Neuroscience
- Giri Panamoottil Krishnan
- Maxim Bazhenov
- Benoît Girard
- Terrence J. Sejnowski
- Mehdi Khamassi
- Hava T. Siegelmann
- Tyler L. Hayes
- Christopher Kanan
In Cognitive Neuroscience
- Maxim Bazhenov
- Giri Panamoottil Krishnan
- Tyler L. Hayes
- Natalia Díaz Rodríguez
- Benoît Girard
- David Filliat
- Timothy M. Hospedales
- Terrence J. Sejnowski
Citations in the corpus
- Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “…further suggests that selective and partial replay in tasks involving reward allow humans to perform generalization and model-based reinforcement learning (Momennejad, 2020), which has inspired several algorithms in machine learning (see Cazé, Khamassi, Aubin, & Girard, 2018, for a review).”
- DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics (2020, methodology): “They may involve the replay of past events to consolidate learning [55, 56] or the exploration of new problem solving strategies [57, 58, 59].”