Rethinking Experience Replay: a Bag of Tricks for Continual Learning

By Pietro Buzzega, Matteo Boschini, Angelo Porrello, Simone Calderara. 2020

185 citations recorded by the source corpus.

In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on classes appearing later in the stream while remaining accurate on previous ones. This is due to the infamous problem of catastrophic forgetting, which causes a quick performance degradation when the classifier focuses on learning new categories. Recent literature proposed various approaches to tackle this issue, often resorting to very sophisticated techniques. In this work, we show that naive rehearsal can be patched to achieve similar performance. We point out some shortcomings that restrain Experience Replay (ER) and propose five tricks to mitigate them. Experiments show that ER, thus enhanced, displays an accuracy gain of 51.2 and 26.9 percentage points on the CIFAR-10 and CIFAR-100 datasets respectively (memory buffer size 1000). As a result, it surpasses current state-of-the-art rehearsal-based methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A soft nearest-neighbor framework for continual semi-supervised learning (2022, neutral): “This is exacerbated by another well-studied phenomenon in CL: overfitting the experience replay buffer [10].”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, methodology): “Lastly, rehearsal methods store and reuse a subset of previously seen data-points to prevent overfitting on current data and avert forgetting [16, 49, 4, 3].”
Exploring Example Influence in Continual Learning (2022, positive): “We set other hyper-settings following ER tricks [4], including 50 total epochs and hyper-parameters.”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “ERT [10] improves ER by balanced sampling and bias control.”
Continual Learning through Retrieval and Imagination (2022, background): “(2) trying to consolidate the knowledge in the original network by replaying a subset of past examples stored in a memory buffer (Buzzega et al. 2020; Hou et al. 2018; Rannen et al. 2017; Rebuffi et al. 2017; Buzzega et al. 2021).”
Transfer without Forgetting (2022, background): “Other works [3,9] propose different sample-selection strategies to include in the buffer, while GEM [37] and its relaxation A-GEM [13] employ old training data to minimize interference.”
Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System (2022, methodology): “We follow the analysis performed in Buzzega et al. (2020b) to observe the probability of each task being predicted at the end of the training.”
Class-Incremental Continual Learning Into the eXtended DER-Verse (2022, background): “, at discrete intervals versus continuously); which elements we keep in memory and which ones we move out [12], [16], [17]; what”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Ebrahimi et al. [102] stored both data instances and their model visual explanations (i.e, saliency maps) in the memory buffer, and encouraged the model to remember the visual explanations for the predictions.”
Continual semi-supervised learning through contrastive interpolation consistency (2021, methodology): “Among all possible approaches, we opt for ER due to its lightweight design and effectiveness [11, 23].”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, methodology): “In addition to ER, we also explore ER with data augmentation (ERaug) [5], ER-MIR [2] and GEM [26] in our experiments.”
Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach (2022, methodology): “Following the analysis in (Buzzega et al., 2021), the normalized probability of each task of a CIFAR-10 trained model is computed by averaging probabilities of all samples belonging to the associated classes in Class-IL setting.”
Continual Learning with Lifelong Vision Transformer (2022, methodology): “We compare LVT with state-of-the-art and well-established methods, including eight rehearsal-based methods (ER [62], GEM [47], AGEM [15], GSS [4], FDR [9], HAL [14], ERT [11], and RM [7]), two meth-ods leveraging Knowledge Distillation (iCaRL [61] and DER++ [10]).”
Provable and Efficient Continual Representation Learning (2022, unknown): “…categorized into three groups (Delange et al. (2021)): replay-based (Lopez-Paz and Ranzato (2017); Rebuﬃ et al. (2017); Rolnick et al. (2018); Buzzega et al. (2021); Aljundi et al. (2019); Borsos et al. (2020)), regularization-based (Kirkpatrick et al. (2017); Zenke et al. (2017); Li and…”
Practical Recommendations for Replay-based Continual Learning Methods (2022, methodology): “Data augmentation in continual learning is explored by Buzzega et al. in [5], in this case, crops and horizontal-ﬂips are applied in the input stream and in the replay buﬀer.”
CaSpeR: Latent Spectral Regularization for Continual Learning (2023, methodology): “Both evaluations improve when applying our regularizer on the evaluated methods.”
Continual Learning of Semantic Segmentation using Complementary 2D-3D Data Representations (2021, methodology): “To update the model parameters θ we use an experience replay strategy, which has shown promising results in prior works [5, 24, 4].”
Convolutional Prompting meets Language Models for Continual Learning (2024, background): “A line of work attempting to maintain a balance, keeps a few samples from previous tasks and uses them for rehearsing the previous concepts while learning a new task [4, 6, 8, 16, 19, 29, 36, 38, 42, 56].”