New Insights on Reducing Abrupt Representation Change in Online Continual Learning

By Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, Eugene Belilovsky. 2021

295 citations recorded by the source corpus.

In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. Experience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes' representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks.

Subfields: Artificial Intelligence, Management Science and Operations Research, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “…performing knowledge distillation [31], [278], improving representations with contrastive learning [276], [279], using asymmetric cross-entropy [280] or constrained gradient directions [45], [63] of the old new training samples, repeated rehearsal with data augmentation [281], properly…”
First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning (2023, background): “While softmax cross entropy loss is considered a standard in classification models, it is shown to be a source of class interference in continual learning [38, 4].”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, methodology): “Following Arani et al. (2022); Buzzega et al. (2020); Cha et al. (2021); Caccia et al. (2021a), we employ ResNet-18 (He et al., 2016) as the backbone to learn a common representation space for all our experiments.”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, neutral): “While both models start with a similar robust decision landscape in τ 0 , later tasks reveal a clear shrinking behavior in ER-ACE.”
Transfer without Forgetting (2022, methodology): “Interestingly, while both ER and ER-ACE do not feature distillation, we find their performance to be competitive for large enough buffers.”
Class-Incremental Continual Learning Into the eXtended DER-Verse (2022, neutral): “As also observed in other recent works [28], [29], [42], this issue can be mitigated by revising the way the cross-entropy loss is applied during training.”
Class-Incremental Learning: A Survey (2023, background): “[236] finds that applying data replay causes the newly added classes’ representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates.”
Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization (2024, methodology): “This analysis is consistent with (Caccia et al., 2022).”
Scaling the Number of Tasks in Continual Learning (2022, background): “In addition, recent papers show that in masking the gradient in the last layer helps classiﬁers to learn continually Caccia et al. [2022], Zeno et al. [2018] even without a supplementary memorization process Lesort et al. [2021a].”
CaSpeR: Latent Spectral Regularization for Continual Learning (2023, methodology): “…our regularizer, we apply it on top of several state-of-the-art rehearsal-based methods: Experience Replay with Asymmetric Cross-Entropy (ER-ACE) [11], Incremental Classifier and Representation Learning (iCaRL) [38], Dark Experience Replay (DER++) [10], eXtended-DER (X-DER) 3 [7] and Pooled…”
Computationally Budgeted Continual Learning: What Does Matter? (2023, background): “Other approaches such as SS-IL [2] and ACE [11,37,61] mask out unaffected classes to reduce their interference during training.”
Online Prototype Learning for Online Continual Learning (2023, background): “PCL clusters samples M times, then regards a representation z of one image (instance) and its cluster center c as a positive pair; z and other cluster centers as negative pairs, formally defined as: where N is the batch size, r is the number of negative samples, and τ m is the temperature…”
Elastic Feature Consolidation for Cold Start Exemplar-free Incremental Learning (2024, methodology): “PR-ACE takes inspiration from the asymmetric loss proposed by Caccia et al. (2021) for online, exemplar-based class incremental learning.”
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, methodology): “We compare our approach with 11 OCL methods (AGEM [9], ER [10], MIR [3], GSS [4], ASER [51], ER-AML [7], GDumb [48], SCR [40], OCM [21], OnPro [67] and GSA [22]) and 7 offline CL methods (DER++ [6], IL2A [75], CO 2 L [8], LUCIR [28], CCIL [45], BIC [69] and SSIL [1]).”
Deep Class-Incremental Learning: A Survey (2023, background): “[248] finds that applying data replay causes the newly added classes’ representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates.”
Improving Plasticity in Online Continual Learning via Collaborative Learning (2023, methodology): “DER++ [6] stores the logits in the memory buffer and extends ER with the distillation of old stored logits.”