Replay in Deep Learning: Current Approaches and Missing Biological Elements

By Tyler L. Hayes, Giri Panamoottil Krishnan, Maxim Bazhenov, Hava T. Siegelmann, Terrence J. Sejnowski, Christopher Kanan. 2021

152 citations recorded by the source corpus.

Abstract Replay is the reactivation of one or more neural patterns that are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated in deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this letter, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be used to improve artificial neural networks.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, Optimization & Training, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In Optimization & Training

In Neuroscience

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “In contrast, the latest surveystypicallycollateonlyalocalaspectofcontinuallearning, withrespecttoitsbiologicalunderpinnings[1],[3],[9],[10], specializedsettingsforvisualclassiﬁcation[11],[12],[13],[14], andspeciﬁcextensionsforNLP[15],[16]orRL[17].”
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks (2022, background): “Also, it does not need periodic retraining on all previously known tasks when learning new concepts (van de Ven et al., 2020; Hayes et al., 2021).”
Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System (2022, background): “Generally, the inspiration from CLS theory in DNNs has been mostly limited to episodic memory and mimicking the rapid and slow learning mechanism is majorly ignored (Hayes et al., 2021) which we aim to address.”
Self-Supervised Training Enhances Online Continual Learning (2021, background): “There are three approaches to mitigating catastrophic forgetting [54]: 1) increasing model capacity to include new representations [2, 46, 47, 56, 58, 59, 61, 66, 67], 2) regularizing parameter updates such that parameters do not deviate too much from their past values [3, 12, 13, 18, 37, 41, 45, 57”
DDGR: Continual Learning with Deep Diffusion-based Generative Replay (2023, methodology): “Among them, the generative replay (GR) model is one of the most popular methods due to its promising experimental results (Hayes et al., 2021).”
Consistent Prompting for Rehearsal-Free Continual Learning (2024, unknown): “Rehearsal [3, 5, 10, 12, 22] is a popular strategy in continual learning, allowing the model to partially access previous exemplars.”
Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation (2022, background): “Our results are in line with a large body of literature suggesting that interleaved training is capable of mitigating catastrophic forgetting in ANNs [4,10,11] and SNNs [12,13], which led to a number of replay-like algorithms involving storing a subset of previous veridical inputs and mixing them wi”
Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2 (2022, methodology): “Different types of replay were discussed by Hayes et al. (2021) along with its role in the brain and in current deep learning implementations.”
Deep Class-Incremental Learning: A Survey (2023, background): “Other surveys either focuses on the specific field [25], [26], [27] or lacking the performance evolution among state-of-the-arts [28], [29], [30].”