Online Continual Learning with Maximally Interfered Retrieval

By Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, Tinne Tuytelaars. 2019

692 citations recorded by the source corpus.

Continual learning, the setting where a learning agent is faced with a never ending stream of data, continues to be a great challenge for modern machine learning systems. In particular the online or "single-pass through the data" setting has gained attention recently as a natural setting that is difficult to tackle. Methods based on replay, either generative or from a stored memory, have been shown to be effective approaches for continual learning, matching or exceeding the state of the art in a number of standard benchmarks. These approaches typically rely on randomly selecting samples from the replay memory or from a generative model, which is suboptimal. In this work, we consider a controlled sampling of memories for replay. We retrieve the samples which are most interfered, i.e. whose prediction will be most negatively impacted by the foreseen parameters update. We show a formulation for this sampling criterion in both the generative replay and the experience replay setting, producing consistent gains in performance and greatly reduced forgetting. We release an implementation of our method at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Infectious Diseases, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Infectious Diseases

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Toselectivelyutilizethe memorybuffer,MIR[145]prioritizestheoldtrainingsamples that subject to more forgetting, while HAL [146] uses them as “anchors” and stabilizes their predictions.”
Learnability and Algorithm for Continual Learning (2023, methodology): “Some methods in this family also study which samples in memory should be used in replay (Aljundi et al., 2019) or which samples in the training data should be saved (Rebuffi et al., 2017; Liu et al., 2020b).”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “Rehearsal with stored data [3–5, 7, 9, 10, 18, 19, 24, 29, 42, 51–53, 64, 68] or samples from a generative model [27, 28,48,56,60] is highly effective when storing training data or training/saving a generative model is possible.”
Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “We compare with several baselines including: finetune that directly trains a classifier on the data stream, GSS [1], Dynamic-OCM [54], MIR [2], Gradient Episodic Memory (GEM) [27], Incremental Classifier and Representation Learning (iCARL) [39], Reservoir [45], CURL, CNDPM, CoPE, ER + GMED and ERa +”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, methodology): “Lastly, rehearsal methods store and reuse a subset of previously seen data-points to prevent overfitting on current data and avert forgetting [16, 49, 4, 3].”
Repeated Augmented Rehearsal: A Simple but Strong Baseline for Online Continual Learning (2022, methodology): “L G ] 13 N ov 2022 be accessed one batch at a time and training is performed with a single pass over the data [Aljundi et al., 2019].”
Exploring Example Influence in Continual Learning (2022, neutral): “Rehearsal-based methods [29, 7, 22, 6, 1, 2, 28, 3, 25] tackle the challenge of SP dilemma by retaining a subset of old tasks in a stored memory buffer with bounded resources.”
Balancing Stability and Plasticity through Advanced Null Space in Continual Learning (2022, neutral): “Replayed-based methods leverage episodic memory to store representative history data or generate virtual data via a generative model, and replay these samples with current data [2, 7, 8, 12, 26, 37, 39, 40, 45, 50, 51].”
Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, methodology): “Maximally Interfering Retrieval (MIR) (Aljundi et al., 2019a), the goal of MIR is to select the examples that are easily forgettable for replay.”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, methodology): “For example, the MIR method (Aljundi et al. 2019a) is a replay-based method for online CL.”
General Incremental Learning with Domain-aware Categorical Representations (2022, background): “Only a few attempts [1, 3, 18] address the general incremental learning problem, but these methods mainly learn from a data stream in an online fashion.”
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning (2022, methodology): “…et al., 2018; von Oswald et al., 2020; Yoon et al., 2018; Li et al., 2019; Xu & Zhu, 2018), experience replay (Lin, 1992; Chaudhry et al., 2019b; Aljundi et al., 2019a; Riemer et al., 2019; Rolnick et al., 2019; Wu et al., 2019), regularization (Kirkpatrick et al., 2017; Zenke et al., 2017;…”
Class-Incremental Continual Learning Into the eXtended DER-Verse (2022, methodology): “It is noted that this practice – which we show in the following opens up interesting possibilities – is not entirely novel to the field: [12], [16] involve a cross-entropy term spanning over all classes (hence, already seen and yet unseen), which silences the activity of future heads (as we discusse”
GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning (2021, neutral): “Maximally Interfered Retrieval (MIR) method [2] uses controlled memory sampling for replay buffer by retrieving samples that the foreseen model parameters update will most negatively impact.”
Continual Learning via Local Module Composition (2021, methodology): “Continual learning methods typically address the problem of forgetting through parameter regularization [46, 70, 99], replay [89, 78, 3, 73, 54, 12, 97, 39, 81, 13] or dynamic architectures (and MoEs) [83, 87, 52, 85, 53, 41].”
Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference (2021, methodology): “To extensively highlight the differences between MIR and CLIB, we ﬁrst have to point out that sample memory methods can be discussed in two aspects: memory management and memory usage.”
Online Continual Learning Via Candidates Voting (2021, methodology): “It ensures that the average loss calculated by using stored old task exemplars does not increase for each training step • MIR refers to Maximally Interfered Retrieval, which applies reservoir sampling to select exemplars for knowledge replay.”