Rehearsal revealed: The limits and merits of revisiting samples in continual learning

By Eli Verwimp, Matthias De Lange, Tinne Tuytelaars. 2021

127 citations recorded by the source corpus.

Learning from non-stationary data streams and overcoming catastrophic forgetting still poses a serious challenge for machine learning research. Rather than aiming to improve state-of-the-art, in this work we provide insight into the limits and merits of rehearsal, one of continual learning’s most established methods. We hypothesize that models trained sequentially with rehearsal tend to stay in the same low-loss region after a task has finished, but are at risk of overfitting on its sample memory, hence harming generalization. We provide both conceptual and strong empirical evidence on three benchmarks for both behaviors, bringing novel insights into the dynamics of rehearsal and continual learning in general. Finally, we interpret important continual learning works in the light of our findings, allowing for a deeper understanding of their successes.1

Subfields: Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “Chaudhry et al. (2019) rebutted against this claim by empirical evidence, while Verwimp et al. (2021) offered empirical counter-evidence for what Chaudhry et al. (2019) argued.”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, critical): “In spite of their widespread application, these approaches fall into a common pitfall: as the memory buffer holds only a small fraction of past examples, there is a high risk of overfitting on that memory [75], thus harming generalization.”
Repeated Augmented Rehearsal: A Simple but Strong Baseline for Online Continual Learning (2022, methodology): “, 2019] and provides conceptual analysis for offline CL [Verwimp et al., 2021], we focus on the online setting and provide theoretical insights through the lens of empirical risk minimization (ERM).”
Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data (2022, background): “is impractical to maintain the training data of each task due to privacy concerns and so forth [4, 11, 20, 22, 29, 30, 32].”
Saliency Guided Experience Packing for Replay in Continual Learning (2021, background): “However, these methods suffer from high forgetting mainly due to overfitting [49] to the small memory samples, thus show suboptimal performance.”
Continual Learning with Lifelong Vision Transformer (2022, methodology): “Recently, various works have focused on the more difficult and realistic class-incremental learning (class-IL) [3, 9, 14, 20, 36, 61, 75, 78, 89, 94], where the network is evaluated on all classes observed during the training, without requiring the task identity.”
Deep Class-Incremental Learning: A Survey (2023, background): “However, since the exemplar set only saves a tiny portion of the training set, data replay may suffer the overfitting problem [59], [134].”
Class-Incremental Learning: A Survey (2023, methodology): “The shade color in the last column denotes the subcategory, which is consistent with Figure 3.”
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, background): “Saving samples of the current task in the buffer could partially solve it, but overstepping the training of buffered data will trigger a severe forgetting problem, namely, overfitting to the memory buffer for old tasks [57, 73].”
Bring Evanescent Representations to Life in Lifelong Class Incremental Learning (2022, methodology): “Most of the successful CIL methods use exemplars of old classes C old to rehearse past knowledge [1, 3, 4, 7, 8, 12, 18, 20, 25, 27, 32, 36].”