Gradient based sample selection for online continual learning
By Rahaf Aljundi, Min Lin, Baptiste Goujaud, Yoshua Bengio. 2019
1,049 citations recorded by the source corpus.
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previously seen data, which is a well known problem of neural networks. To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal. Previous works often depend on task boundary and i.i.d. assumptions to properly select samples for the replay buffer. In this work, we formulate sample selection as a constraint reduction problem based on the constrained optimization view of continual learning. The goal is to select a fixed subset of constraints that best approximate the feasible region defined by the original constraints. We show that it is equivalent to maximizing the diversity of samples in the replay buffer with parameters gradient as the feature. We further develop a greedy alternative that is cheap and efficient. The advantage of the proposed method is demonstrated by comparing to other alternatives under the continual learning setting. Further comparisons are made against state of the art methods that rely on task boundaries which show comparable or even better results for our method.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Bernt Schiele
- Xiaogang Wang
- Andrea Vedaldi
- Andrew Zisserman
In Optimization & Training
- Frederik Benzing
- Tinne Tuytelaars
- Alexander Rainer Tassilo Gepperth
- Joost van de Weijer
- Rahaf Aljundi
- Fei Ye
- Adrian G. Bors
- Tyler L. Hayes
In Meta & Continual Learning
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Correspondingly, GCL consists of two interconnected settings: Task-Free Continual Learning (TFCL) [20], where the task identities are not accessible in either training or testing; and Online Continual Learning (OCL) [21], where the training samples are observed in an one-pass data stream.”
- The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “…and more challenging case of task-agnostic or task-free continual learning, where in the training phase the learner has no access to task identities (Zeno et al., 2018; 2021; Aljundi et al., 2019a; Lee et al., 2020; Jin et al., 2021; Wang et al., 2022c; Pourcel et al., 2022; Ye & Bors, 2022).”
- Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “The Gradient Sample Selection (GSS) [1] is another sample selection approach that treats sample selection as a constrained optimization reduction.”
- On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, neutral): “However, recent works put an increasing focus on the harder Class-Incremental setting (CIL) [4, 32, 15, 81], which entails the production of a unified prediction encompassing all seen classes.”
- Continual Learning by Modeling Intra-Class Variation (2022, methodology): “We perform the experiments on CIFAR-100 (5 tasks with 20 classes per task) and TinyImageNet (10 tasks with 20 classes per task) with buffer size 500.”
- Exploring Example Influence in Continual Learning (2022, neutral): “Rehearsal-based methods [29, 7, 22, 6, 1, 2, 28, 3, 25] tackle the challenge of SP dilemma by retaining a subset of old tasks in a stored memory buffer with bounded resources.”
- Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer (2022, methodology): “Exemplar-based methods [42, 5, 51, 33, 6] sample a very small subset of each task data et ∈ τt at the end of every task τt and store it in a memory buffer Mt = {e1, e2, · · · , et}, which contains a subset of data from all tasks seen until time t.”
- Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “We compare CVT with state-of-the-art and well-established Online CL baselines, including 11 rehearsal-based methods (ER [63], GEM [49], AGEM [14], GSS [4], FDR [8], HAL [13], ASERμ [67], ERT [10], RM [6], SCR [52], and CLS [5]), 2 methods leveraging Knowledge Distillation (iCaRL [62] and DER++ [9]).”
- Incremental Task Learning with Incremental Rank Updates (2022, background): “Incremental task learning is a subcategory of continual learning or lifelong learning approaches aim to address the problem of catastrophic forgetting by adapting the network or training process to learn new tasks without forgetting the previously learned ones [23,16,3,2,4,6,28,29,12].”
- Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, methodology): “Maximally Interfering Retrieval (MIR) (Aljundi et al., 2019a), the goal of MIR is to select the examples that are easily forgettable for replay.”
- Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “For example, the MIR method (Aljundi et al. 2019a) is a replay-based method for online CL.”
- Continual Learning through Retrieval and Imagination (2022, background): “GSS (Aljundi et al. 2019) introduces a gradient-based sampling to store optimally selected exemplars in the memory buffer.”
- Transfer without Forgetting (2022, background): “The former scenario is a relaxation of the latter, as it provides the model with the task identifier of each sample at test time; for this reason, we focus our evaluation mainly on the ClassIL protocol, highlighted as a more realistic and challenging benchmark [20,3].”
- Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “However, when moving to lifelong learning setting, their performance significantly drops (Kirkpatrick et al., 2017; Aljundi et al., 2019; Biesialska et al., 2020; Cui et al., 2021; Ke et al., 2021b; Madotto et al., 2021; Ke et al., 2021a; Feng et al., 2022).”
- Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “We also include the feature-space clustering method (FSS-Clust) (Aljundi et al., 2019b) for Permuted MNIST and Split MNIST.”
- Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation (2022, methodology): “Also, [4] identifies an exemplar set for the best approximation of the feasible region in the model parameter space induced by the whole data.”
- Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, background): “Recent studies [2, 5, 27] argue that the disjoint CL setup is less realistic but prevalent in literature and propose a new setup where task boundaries share classes, called ‘blurry’.”
- The Challenges of Continuous Self-Supervised Learning (2022, methodology): “The training data stream is created by sampling images at random from the four datasets {Dp1 , Dp2 , Dp3 , Dp4 } where [ p1 , p2 , p3 , p4] is a permutation of the sequence [1 , 2 , 3 , 4] .”