Gradient-based Editing of Memory Examples for Online Task-free Continual Learning

By Xisen Jin, Arka Sadhu, Junyi Du, Xiang Ren. 2020

117 citations recorded by the source corpus.

We explore task-free continual learning (CL), in which a model is trained to avoid catastrophic forgetting in the absence of explicit task boundaries or identities. Among many efforts on task-free CL, a notable family of approaches are memory-based that store and replay a subset of training examples. However, the utility of stored seen examples may diminish over time since CL models are continually updated. Here, we propose Gradient based Memory EDiting (GMED), a framework for editing stored examples in continuous input space via gradient updates, in order to create more"challenging"examples for replay. GMED-edited examples remain similar to their unedited forms, but can yield increased loss in the upcoming model updates, thereby making the future replays more effective in overcoming catastrophic forgetting. By construction, GMED can be seamlessly applied in conjunction with other memory-based CL algorithms to bring further improvement. Experiments validate the effectiveness of GMED, and our best method significantly outperforms baselines and previous state-of-the-art on five out of six datasets. Code can be found at https://github.com/INK-USC/GMED.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Besides, the old training samples can be continually adjusted to accommodate incremental changes, e.g., making them more representative [143] or challenging [144] for separation.”
The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “…and more challenging case of task-agnostic or task-free continual learning, where in the training phase the learner has no access to task identities (Zeno et al., 2018; 2021; Aljundi et al., 2019a; Lee et al., 2020; Jin et al., 2021; Wang et al., 2022c; Pourcel et al., 2022; Ye & Bors, 2022).”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “For instance, memory-based approaches can store a few past samples from the data stream at each training time and replay them later during the training [8, 12].”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, neutral): “The authors of [83] reported that, theoretically, no correlation exists between the Hessian w.r.t. weights and the robustness of the model w.r.t. the input.”
Exploring Example Influence in Continual Learning (2022, neutral): “We compare our method against 8 rehearsal-based methods (including GDUMB [28], GEM [22], AGEM [6], HAL [5], GSS [2], MIR [1], GMED [33] and ER [7]).”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “We consider a strict evaluation setting [38,75] for online continual learning, including task-aware protocol [56] and task-free protocol [39].”
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, methodology): “The CL model would overfit the memory buffer, and this approach could be gradually less effective for mitigating forgetting as the model repeatedly learns the memory buffer (Jin et al., 2021), illustrated in Figure 1 (a).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Jin et al. (2021) leveraged this idea by directly modifying the samples in memory via gradient updates to make them more challenging for the learner.”
A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning (2022, background): “There are other methods that do not fall into these two groups [24, 29, 23, 46, 61, 66], and we refer the readers to [10, 36, 62] for a holistic review.”
Deep Class-Incremental Learning: A Survey (2023, background): “Future class-incremental learning algorithms should be able to conduct fully online training [226], [227], [228], [229], [230] without requiring the task boundaries [231], [232], [233].”
Learning an Evolved Mixture Model for Task-Free Continual Learning (2022, methodology): “methods under this challenging dataset, where the other methods results are cited from [25] and where MIR is the Maximally Interfered Retrieval.”
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, background): “…size M to determine the number of old training samples that can be stored in the memory buffer; (2) memory retrieval strategy [3, 51] to gather old training samples from the memory buffer; and (3) memory update strategy [4, 23, 30] to keep an appropriate data structure across tasks and classes.”
On the Soft-Subnetwork for Few-shot Class Incremental Learning (2022, methodology): “According to the access to task identity during the training and inference phase, the research is often categorized into specific problems, such as task-incremental [34, 8], class-incremental [4, 25, 27, 37, 23, 3, 19, 46], and task-free continual learning [1, 21, 35, 11].”
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning (2024, background): “GMED [17] edits the exemplars by monitoring the change in loss when training on incoming data.”
Retrospective Adversarial Replay for Continual Learning (2022, methodology): “GMED [25] uses gradient updates to edit buffered samples individually based on their loss increase after a look-ahead update performed only on the incoming batch.”