Rainbow Memory: Continual Learning with a Memory of Diverse Samples

By Jihwan Bang, Heesu Kim, Y. Yoo, Jung-Woo Ha, Jonghyun Choi. 2021

485 citations recorded by the source corpus.

Continual learning is a realistic learning scenario for AI models. Prevalent scenario of continual learning, however, assumes disjoint sets of classes as tasks and is less realistic rather artificial. Instead, we focus on ‘blurry’ task boundary; where tasks shares classes and is more realistic and practical. To address such task, we argue the importance of diversity of samples in an episodic memory. To enhance the sample diversity in the memory, we propose a novel memory management strategy based on per-sample classification uncertainty and data augmentation, named Rainbow Memory (RM). With extensive empirical validations on MNIST, CIFAR10, CIFAR100, and ImageNet datasets, we show that the proposed method significantly improves the accuracy in blurry continual learning setups, outperforming state of the arts by large margins despite its simplicity. Code and data splits will be available in https://github.com/clovaai/rainbow-memory.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “BBCL [28], [46] {Dt, t}t∈T ; p(Xi) 6= p(Xj), Yi 6= Yj and Yi ∩ Yj 6= ∅ for i 6= j {p(Xt)}t∈T ; t is unavailable”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “…2017; Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Chaudhry et al., 2019; Aljundi et al., 2019b; Buzzega et al., 2020; Verwimp et al., 2021; Bang et al., 2021; Kim et al., 2022; Zhang et al., 2022; Wang et al., 2022a; Saha & Roy, 2023), expansion-based methods (Rusu et al., 2016; Yoon et…”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “Rehearsal with stored data [3–5, 7, 9, 10, 18, 19, 24, 29, 42, 51–53, 64, 68] or samples from a generative model [27, 28,48,56,60] is highly effective when storing training data or training/saving a generative model is possible.”
A Theoretical Study on Solving Continual Learning (2022, methodology): “Note that the CIL definition and the subsequent analysis are applicable to tasks with any number of classes (including only one class per task) and to online CIL where the training data for each task or class comes gradually in a data stream and may also cross task boundaries (blurry tasks [68]) bec”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Another kind of approaches for continual learning focuses on the memory system, which usually employs a small memory buffer [3, 5, 6, 28, 36, 44, 59] for storing some past data or trains a generator to provide the replay samples when learning new tasks [38, 43, 46, 47, 58, 57, 52].”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, positive): “Several approaches mitigate such an issue through dataaugmentation techniques, either by generating different versions of the same buffer datapoint [7, 15] or by combining different examples into a single one [14, 11].”
Repeated Augmented Rehearsal: A Simple but Strong Baseline for Online Continual Learning (2022, background): “, 2019, Cha et al., 2021]. The online CL setting is more challenging because the agent can only access the current batch of incoming data and performs training with a single pass through the data. Experience Replay (Rehearsal): Chaudhry et al. (2019) propose experience replay (ER), which performs jo”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “We compare CVT with state-of-the-art and well-established Online CL baselines, including 11 rehearsal-based methods (ER [63], GEM [49], AGEM [14], GSS [4], FDR [8], HAL [13], ASERμ [67], ERT [10], RM [6], SCR [52], and CLS [5]), 2 methods leveraging Knowledge Distillation (iCaRL [62] and DER++ [9]).”
Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data (2022, methodology): “Previous works widely adopt rehearsal strategy [3, 6, 16, 22, 32]: storing a limited quantity of samples called exemplars from the original training dataset and reusing them against forgetting when the model”
Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “However, the majority of approaches focus on task-based continual learning and update the memory relying on task boundaries (Rebuffi et al., 2017; Nguyen et al., 2018; Titsias et al., 2020; Pan et al., 2020; Bang et al., 2021; Yoon et al., 2021).”
Class-Incremental Learning with Strong Pre-trained Models (2022, methodology): “Even Rainbow Memory [5] which explicitly tackles fully overlapping classes, underperforms, partially because it assumes the distribution of each overlapping class does not change.”
Constrained Few-shot Class-incremental Learning (2022, methodology): “To address the catastrophic forgetting issue in CIL, rehearsal methods store some old training data, and replay them while learning novel classes during the incremental stage [1, 3, 31, 32, 36, 39].”
Representation Compensation Networks for Continual Semantic Segmentation (2022, background): “The knowledge can be stored by multiple types, like examples [4, 6, 9, 11, 73, 83], prototypes [35, 106, 107], generative networks [60], etc.”
Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference (2021, methodology): “Recently, to make the setup more realistic, a blurry task setup has been proposed and investigated (Aljundi et al., 2019c; Prabhu et al., 2020; Bang et al., 2021), where 100 − M % of the sampels are from the dominant class of the task and M % of the samples are from all classes, where M is the…”
Towards Continual Knowledge Learning of Language Models (2021, neutral): “This result also highlights the main difference between CKL and CL; while rehearsal methods show strong performances in traditional CL settings (Prabhu et al., 2020; Bang et al., 2021), in CKL, it shows the worst performance since the update of outdated knowledge and acquisition of new knowledge is ”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[87] proposed RainbowMemory (RM) to select previous data instances, particularly in the scenario where different tasks can share the same classes.”
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021, methodology): “To incorporate these corpora, research on pre-training under continual learning setup (Bang et al., 2021) is required.”
Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners (2022, methodology): “To overcome catastrophic forgetting in artificial neural networks, three types of methods have been explored in the past few years, replay [15, 3, 6, 20, 10], regularization [9, 19, 8, 4, 17, 13, 5, 7, 16], and expand [22, 14, 1, 21].”