Brain-inspired replay for continual learning with artificial neural networks

By Gido M. van de Ven, Hava T. Siegelmann, Andreas S. Tolias. 2020

604 citations recorded by the source corpus.

Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those memories. In artificial neural networks, such memory replay can be implemented as ‘generative replay’, which can successfully – and surprisingly efficiently – prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario. However, scaling up generative replay to complicated problems with many tasks or complex inputs is challenging. We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network’s own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks (e.g., class-incremental learning on CIFAR-100) without storing data, and it provides a novel model for replay in the brain. One challenge that faces artificial intelligence is the inability of deep neural networks to continuously learn new information without catastrophically forgetting what has been learnt before. To solve this problem, here the authors propose a replay-based algorithm for deep learning without the need to store data.

Subfields: Artificial Intelligence, Meta & Continual Learning, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

In Neuroscience

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “BI-R [412] incorporates context-modulated feedback connections in a standard VAE to replay internal”
Continual Learning through Retrieval and Imagination (2022, methodology): “BIR (van de Ven et al. 2020) replays hidden representations generated by the network’s own feedback connections.”
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks (2022, background): “Also, it does not need periodic retraining on all previously known tasks when learning new concepts (van de Ven et al., 2020; Hayes et al., 2021).”
Continual Learning with Guarantees via Weight Interval Constraints (2022, background): “…& Ranzato, 2017; Shin et al., 2017; Zenke et al., 2017; Aljundi et al., 2018; Masse et al., 2018; Liu et al., 2018; Rolnick et al., 2019; van de Ven et al., 2020), they typically do not provide any firm guarantees about the degree of forgetting experienced by the model, which renders them…”
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “Continual learning in practice The study of continual learning (or lifelong learning) dates back to the work of [34] and it receives a surge of research interest over recent years [14, 17, 10, 5, 13, 29, 32, 16, 27, 37].”
Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning (2021, background): “Humans have the ability to continually learn new knowledge without forgetting their previously learned ones through mediating a rich set of neurocognitive mechanisms [41, 15, 39].”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, methodology): “We are not surprised to see that LwF [36] performs worse than naive rehearsal, as this is also common for class-incremental learning [57, 56].”
Natural continual learning: success is a journey, not (just) a destination (2021, methodology): “Instead, these two approaches have been found to further improve robustness to catastrophic forgetting when combined [31, 44].”
Model Zoo: A Growing Brain That Learns Continually (2021, methodology): “…2017; Lee et al., 2017), new architectures (Serra et al., 2018), generative replay-based (Mocanu et al., 2016; Shin et al., 2017; Liu et al., 2020; Ven et al., 2020), ensemble-based (Aljundi et al., 2017; Wen et al., 2020) and methods that select locally-redundant directions in the weight space…”
Rehearsal revealed: The limits and merits of revisiting samples in continual learning (2021, background): “Although promising results for generative models have been reported in conﬁned setups[40], learning both the encoder and decoder in a continual fashion can be cumbersome.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “…in some recent works that replay high-level feature representations (Hayes, Kafle, Shrestha, Acharya, & Kanan, 2020; Iscen, Zhang, Lazebnik, & Schmid, 2020; Caccia, Belilovsky, Caccia, & Pineau, 2020; Pellegrini, Graffieti, Lomonaco, & Maltoni, 2019; van de Ven, Siegelmann, & Tolias, 2020).”
Distilling Causal Effect of Data in Class-Incremental Learning (2021, background): “In fact, even though that the ever-increasing extra storage in data replay contradicts with the requirement of CIL, it is still the most reliable solution for antiforgetting [35, 45, 5].”
Posterior Meta-Replay for Continual Learning (2021, methodology): “This approach is in contrast with how PriorFocused methods like EWC are commonly deployed in task-agnostic inference settings, where the softmax output grows as new tasks arrive (e.g., [73]).”
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, methodology): “…pseudo-labelled data corresponding to the prior task is generated and interleaved with data from the current task to train a solver whereas van de Ven et al. (2020) use the same model as both a solver and generator and instead of replaying actual samples, replay latent representations. et al.…”
The Effectiveness of Memory Replay in Large Scale Continual Learning (2020, background): “Another line of research uses generative models to construct replay examples (Shin et al., 2017; van de Ven et al., 2020).”
Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners (2022, methodology): “To overcome catastrophic forgetting in artificial neural networks, three types of methods have been explored in the past few years, replay [15, 3, 6, 20, 10], regularization [9, 19, 8, 4, 17, 13, 5, 7, 16], and expand [22, 14, 1, 21].”