Online Coreset Selection for Rehearsal-based Continual Learning

By Jaehong Yoon, Divyam Madaan, Eunho Yang, Sung Ju Hwang. 2021

187 citations recorded by the source corpus.

A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a current dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “…[21], performance of corresponding tasks with cardinality constraints [132], mini-batch gradient similarity and cross-batch gradient diversity [133], ability of optimizing latent decision boundaries [134], diversity of robustness against perturbations [32], similarity to the gradients of old…”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “Another kind of approaches for continual learning focuses on the memory system, which usually employs a small memory buffer [3, 5, 6, 28, 36, 44, 59] for storing some past data or trains a generator to provide the replay samples when learning new tasks [38, 43, 46, 47, 58, 57, 52].”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, neutral): “Other works [3, 4, 16, 85], instead, select carefully the valuable samples that should be inserted into the buffer: they argue that random selection may pick non-informative and noisy instances, affecting the model generalization.”
Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “However, the majority of approaches focus on task-based continual learning and update the memory relying on task boundaries (Rebuffi et al., 2017; Nguyen et al., 2018; Titsias et al., 2020; Pan et al., 2020; Bang et al., 2021; Yoon et al., 2021).”
Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, background): “Rainbow memory [5] and OCS [37] are proposed to diversified sampling by regular interval of uncertainty and constructing a ‘Core-Set’ [31], respectively.”
GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning (2021, neutral): “However, previous approaches to coreset in continual learning have focused on qualitative/diversity-based criteria [8, 9], or bi-level optimizations with signiﬁcant computational costs and scaling limitations [10].”
On the Soft-Subnetwork for Few-shot Class Incremental Learning (2022, unknown): “OCS [50] tackles the class imbalance problems for rehearsal-based continual learning, where the number of instances at each class varies per task, so the model would perform biased training on ar X iv :2 20 9.”
Computationally Budgeted Continual Learning: What Does Matter? (2023, background): “For example, Herding [47], K-Means [16], OCS [60], InfoRS [52], RM [6], and GSS [5] aim to maximize diversity among samples selected for training with different metrics.”
Retrospective Adversarial Replay for Continual Learning (2022, methodology): “To retain the knowledge of previous tasks and mitigate catastrophic forgetting, similar to [2, 55, 65], we utilize a limited memory buffer to store samples representing previously seen data.”