Generative replay with feedback connections as a general strategy for continual learning

By Gido M. van de Ven, Andreas S. Tolias. 2018

253 citations recorded by the source corpus.

A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but differences in evaluation protocols make it difficult to directly compare their performance. To enable more meaningful comparisons, here we identified three distinct scenarios for continual learning based on whether task identity is known and, if it is not, whether it needs to be inferred. Performing the split and permuted MNIST task protocols according to each of these scenarios, we found that regularization-based approaches (e.g., elastic weight consolidation) failed when task identity needed to be inferred. In contrast, generative replay combined with distillation (i.e., using class probabilities as "soft targets") achieved superior performance in all three scenarios. Addressing the issue of efficiency, we reduced the computational cost of generative replay by integrating the generative model into the main model by equipping it with generative feedback or backward connections. This Replay-through-Feedback approach substantially shortened training time with no or negligible loss in performance. We believe this to be an important first step towards making the powerful technique of generative replay scalable to real-world continual learning applications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Meta & Continual Learning

Citations in the corpus

Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, methodology): “For comparison, we pick several popular methods (EWC[3], online EWC[23], SI[9], MAS[24], LwF[25]) and state-of-the-art rehearsal-based methods (DGR[8], RtF[14]) with generative model.”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, methodology): “Regularizing knowledge in the prediction space is done using knowledge distillation [22] and it has been found to perform better than model regularization-based methods for continual learning when task labels are not given [36, 60].”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Typically, most generative replay based methods [128, 130] generate only previous data instances along with their corresponding labels.”
Natural continual learning: success is a journey, not (just) a destination (2021, methodology): “While the online Laplace approximation has been applied successfully in several continual learning settings (Kirkpatrick et al., 2017; Ritter et al., 2018), it has also been found to perform sub-optimally on a range of problems (van de Ven and Tolias, 2018; Duncker et al., 2020).”
Posterior Meta-Replay for Continual Learning (2021, methodology): “While in practice this approach still seems to be out of reach for high-dimensional problems [52], generative models for data replay have been successfully applied to CL [66, 71, 75].”
Online Continual Learning in Image Classification: An Empirical Survey (2021, methodology): “Instead of saving raw samples, an alternative is Generative Replay which trains a deep generative model such as GAN [45] to generate pseudodata that mimic past data for replay [46, 47, 48].”
Do not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting (2021, methodology): “Other data-generation based methods [23, 33, 38, 39, 42] use generative adversarial network [9] and by replying the total data, new model is trained.”
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams (2020, background): “To date, the paradigms of task, class, and domain incremental learning [46] dominate the continual learning literature.”
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent (2020, methodology): “The idea behind memory-based methods is to store data from previous tasks in a buffer of fixed size, which can then be reused during training on the current task (Chaudhry et al. 2019, Van de Ven & Tolias 2018).”
GAN Memory with No Forgetting (2020, methodology): “Testing classification accuracy on all 36 categories from the compared methods along training are summarized in Figure 6 (right), It’s clear that EWC barely works in the class-incremental learning scenario [77, 40, 78].”
Continual Learning with Adaptive Weights (CLAW) (2019, neutral): “…data (or pseudo-data) from previous tasks (Ratcliff, 1990; Robins, 1993; 1995; Thrun, 1996; Schmidhuber, 2013; Hattori, 2014; Mocanu et al., 2016; Rebuffi et al., 2017; Kamra et al., 2017; Shin et al., 2017; Rolnick et al., 2018; van de Ven & Tolias, 2018; Wu et al., 2018; Titsias et al., 2019).”
Continual Unsupervised Representation Learning (2019, methodology): “Another popular approach is that of rehearsal-based methods (Robins, 1995), where the data distribution from earlier tasks is captured by samples from a generative model trained concurrently (Shin et al., 2017; van de Ven & Tolias, 2018; Ostapenko et al., 2018).”
Continual learning with hypernetworks (2019, methodology): “For longer task sequences ( T = 100 ), SI and DGR+distill (Shin et al., 2017; van de Ven & Tolias, 2018) degrade gracefully, while the regularization strength of online EWC prevents the method from achieving high accuracy (see Fig.”
Improving and Understanding Variational Continual Learning (2019, methodology): “More recently, this idea has been made more efﬁcient by integrating the generative model into the training procedure [21].”
Three scenarios for continual learning (2019, methodology): “For a further discussion of the scalability of generative replay we refer to [10].”