Reducing Representation Drift in Online Continual Learning

21 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Continual Learning by Modeling Intra-Class Variation (2022, methodology): “A more memory-friendly setting is memory-based online continual learning (e.g., (Caccia et al., 2021)), which has the same training objective as Equation 2 but all the training data can only be sampled and used once.”
Continual Learning via Local Module Composition (2021, methodology): “Continual learning methods typically address the problem of forgetting through parameter regularization [46, 70, 99], replay [89, 78, 3, 73, 54, 12, 97, 39, 81, 13] or dynamic architectures (and MoEs) [83, 87, 52, 85, 53, 41].”
Continual Learning in Deep Networks: an Analysis of the Last Layer (2021, methodology): “[19, 5] proposed to apply a cosine normalization of the layer to avoid the norm and bias unbalance.”
Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach (2022, background): “…better across tasks than supervised pre-training in CL. Owing to additional computational effort, some of the approaches, e.g. (Zhang et al., 2020; Caccia et al., 2021; Mazumder et al., 2021; Su et al., 2020), relinquished pre-training altogether and employed auxiliary pretext task to boost…”
Dealing with Cross-Task Class Discrimination in Online Continual Learning (2023, background): “ERAML [7] avoids CF by shielding the learned representations from drastic adaptation to accommodate new classes.”