The Effectiveness of Memory Replay in Large Scale Continual Learning

By Yogesh Kumar Balaji, Mehrdad Farajtabar, Dong Yin, Alex Mott, Ang Li. 2020

41 citations recorded by the source corpus.

We study continual learning in the large scale setting where tasks in the input sequence are not limited to classification, and the outputs can be of high dimension. Among multiple state-of-the-art methods, we found vanilla experience replay (ER) still very competitive in terms of both performance and scalability, despite its simplicity. However, a degraded performance is observed for ER with small memory. A further visualization of the feature space reveals that the intermediate representation undergoes a distributional drift. While existing methods usually replay only the input-output pairs, we hypothesize that their regularization effect is inadequate for complex deep models and diverse tasks with small replay buffer size. Following this observation, we propose to replay the activation of the intermediate layers in addition to the input-output pairs. Considering that saving raw activation maps can dramatically increase memory and compute cost, we propose the Compressed Activation Replay technique, where compressed representations of layer activation are saved to the replay buffer. We show that this approach can achieve superior regularization effect while adding negligible memory overhead to replay method. Experiments on both the large-scale Taskonomy benchmark with a diverse set of tasks and standard common datasets (Split-CIFAR and Split-miniImageNet) demonstrate the effectiveness of the proposed method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Pediatrics, Perinatology and Child Health, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “In contrast, reservoir sampling (Vitter, 1985) is used (Rolnick et al., 2019; Chaudhry et al., 2019b; Buzzega et al., 2020; Isele & Cosgun, 2018; Balaji et al., 2020) to update the memory online and maintain uniform samples from the stream.”
Architecture Matters in Continual Learning (2022, background): “…Riemer et al., 2018) or incorporate them as part of the optimization process (e.g., gradient projection) (Farajtabar et al., 2020; Chaudhry et al., 2018; Balaji et al., 2020), or learn a generative model (Shin et al., 2017; Kirichenko et al., 2021) or kernel (Derakhshani et al., 2021) to do so.”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, background): “This regularization can also take place in the feature space rather than the prediction space [4, 22], which we refer to as feature distillation.”
Same State, Different Task: Continual Reinforcement Learning without Interference (2021, background): “2019) and has been shown to be a very effective baseline in CL (Balaji et al. 2020).”
Rainbow Memory: Continual Learning with a Memory of Diverse Samples (2021, methodology): “However, those approaches overlook the importance of memory management and normally adopt simple random sampling [17, 2] or reservoir sampling [40] or ring-buffer sampling [7].”
TiC-CLIP: Continual Training of CLIP Models (2023, unknown): “Simple replay-based baselines can surpass various methods on standard benchmarks (Lomonaco et al., 2022; Balaji et al., 2020; Prabhu et al., 2020).”
Class-Incremental Learning with Generative Classifiers (2021, background): “The size of this memory buffer is typically one of the most important determinants of a method’s performance [4, 42].”
One Pass ImageNet (2021, background): “Existing works have shown that memory replay is effective in sequential learning [11, 4].”
Reducing Representation Drift in Online Continual Learning (2021, background): “Lastly, (Balaji et al., 2020) concurrently argue that replay based methods undergo distributional shifts in latent space.”