Repeated Augmented Rehearsal: A Simple but Strong Baseline for Online Continual Learning

By Yaqian Zhang, Bernhard Pfahringer, Eibe Frank, Albert Bifet, Nick Jin Sean Lim, Yunzhe Jia. 2022

56 citations recorded by the source corpus.

Online continual learning (OCL) aims to train neural networks incrementally from a non-stationary data stream with a single pass through data. Rehearsal-based methods attempt to approximate the observed input distributions over time with a small memory and revisit them later to avoid forgetting. Despite its strong empirical performance, rehearsal methods still suffer from a poor approximation of the loss landscape of past data with memory samples. This paper revisits the rehearsal dynamics in online settings. We provide theoretical insights on the inherent memory overfitting risk from the viewpoint of biased and dynamic empirical risk minimization, and examine the merits and limits of repeated rehearsal. Inspired by our analysis, a simple and intuitive baseline, Repeated Augmented Rehearsal (RAR), is designed to address the underfitting-overfitting dilemma of online rehearsal. Surprisingly, across four rather different OCL benchmarks, this simple baseline outperforms vanilla rehearsal by 9%-17% and also significantly improves state-of-the-art rehearsal-based methods MIR, ASER, and SCR. We also demonstrate that RAR successfully achieves an accurate approximation of the loss landscape of past data and high-loss ridge aversion in its learning trajectory. Extensive ablation studies are conducted to study the interplay between repeated and augmented rehearsal and reinforcement learning (RL) is applied to dynamically adjust the hyperparameters of RAR to balance the stability-plasticity trade-off online. Code is available at https://github.com/YaqianZhang/RepeatedAugmentedRehearsal

Subfields: Cognitive Neuroscience, Education, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Ziwei Liu

In Education

1. De-chuan Zhan
2. Han-Jia Ye

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Qiwen Wang
5. Ziwei Liu
6. Zhi-Hong Qi
7. René Vidal
8. Liyuan Wang

Citations in the corpus

• The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “(18) Rehearsal has shown good empirical performance in deep continual learning; see, e.g., Chaudhry et al. (2019); Prabhu et al. (2020); Zhang et al. (2022); Bonicelli et al. (2022).”
• Deep Class-Incremental Learning: A Survey (2023, background): “Future class-incremental learning algorithms should be able to conduct fully online training [226], [227], [228], [229], [230] without requiring the task boundaries [231], [232], [233].”
• Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, background): “While recent strides have been made with the help of data augmentation [73, 75] and contrastive representation learning [12, 38, 40, 71], short-comings such as the sub-optimality of convergence and the demand for large batch size with higher computational overhead remind us a chasm in performance…”