Using Hindsight to Anchor Past Knowledge in Continual Learning

By Arslan Chaudhry, Albert Gordo, Puneet Kumar Dokania, Philip H. S. Torr, David Lopez-Paz. 2019

289 citations recorded by the source corpus.

In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of experience replay, re-learning on past data stored in a small buffer known as episodic memory. In this work, we complement experience replay with a new objective that we call ``anchoring'', where the learner uses bilevel optimization to update its knowledge on the current task, while keeping intact the predictions on some anchor points of past tasks. These anchor points are learned using gradient-based optimization to maximize forgetting, which is approximated by fine-tuning the currently trained model on the episodic memory of past tasks. Experiments on several supervised learning benchmarks for continual learning demonstrate that our approach improves the standard experience replay in terms of both accuracy and forgetting metrics and for various sizes of episodic memory.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Toselectivelyutilizethe memorybuffer,MIR[145]prioritizestheoldtrainingsamples that subject to more forgetting, while HAL [146] uses them as “anchors” and stabilizes their predictions.”
Learnability and Algorithm for Continual Learning (2023, methodology): “, 2020), HAL (Chaudhry et al., 2021), and MORE (Kim et al.”
On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, positive): “At ﬁrst glance, the regularization of our approach can be understood as a mean to enforce ﬂat minima for each of the tasks, as advocated by Mirzadeh et al. [54] and Yin et al. [84].”
Exploring Example Influence in Continual Learning (2022, neutral): “We compare our method against 8 rehearsal-based methods (including GDUMB [28], GEM [22], AGEM [6], HAL [5], GSS [2], MIR [1], GMED [33] and ER [7]).”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “Recent studies have focused on a more practical Task-free protocol, which evaluates the network on all classes observed during training without requiring the identity of the task [3,81,8,95,37,90,13].”
Continual Learning through Retrieval and Imagination (2022, background): “Unfortunately, most of the exiting methods fail to satisfy all these guidelines mentioned above (Chaudhry et al. 2021; Wang et al. 2020; Lopez-Paz et al. 2017; Wei et al. 2021; Riemer et al. 2019; Kong et al. 2021; Schwarz et al. 2018; Zhu et al. 2020).”
Transfer without Forgetting (2022, background): “Other works [3,8] propose different sampleselection strategies to include in the buffer, while GEM [33] and its relaxed version A-GEM [9] employ old training data to minimize interference in an explicit manner.”
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, background): “Due to their conceptual simplicity, generalizability to various settings, and superior ability to mitigate catastrophic forgetting, rehearsal-based methods have been widely recognized as the reigning state-of-the-art [5, 3] in the challenging class-incremental setting.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, background): “The knowledge can be stored by multiple types, like examples [4, 6, 9, 11, 73, 83], prototypes [35, 106, 107], generative networks [60], etc.”
Wide Neural Networks Forget Less Catastrophically (2021, methodology): “Perhaps the strongest are the replay-based methods that store a small subset of data from previous tasks in the replay buffer, and either use it directly in the form of experience replay with new task (Shin et al., 2017; Rolnick et al., 2018; Riemer et al., 2018; Chaudhry et al., 2019; Chaudhry et a”
Online Continual Learning Via Candidates Voting (2021, methodology): “(1) For methods using original data as exemplars [34, 2, 3, 29, 8, 9, 7, 35, 6, 42, 45, 13], the storage requirement for storing data in exemplar set is O(3 × S(2) × Q × C).”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “They used selective memory retrieval, which is shown to be more effective compared to random retrieval in continual analogical reasoning.”
Continual semi-supervised learning through contrastive interpolation consistency (2021, background): “the grounds that real-world CL systems would never see the same input twice [10, 11, 12].”
Co2L: Contrastive Continual Learning (2021, methodology): “Following this simple setup, ER-based methods mainly focus on either regulating model updates not to contradict the learning objectives on past samples [29, 5] or selecting the most representative/forgetting-prone samples to prevent changes in past predictions [2, 7, 33].”
ACAE-REMIND for Online Continual Learning with Compressed Feature Replay (2021, methodology): “And we compare with online continual learning methods: AGEM [5], BGD [36], GEM [22], GSS-Greedy [3], HAL [4], ER [31], MIR [1], and GMED [14].”
Rainbow Memory: Continual Learning with a Memory of Diverse Samples (2021, methodology): “However, those approaches overlook the importance of memory management and normally adopt simple random sampling [17, 2] or reservoir sampling [40] or ring-buffer sampling [7].”
Continual Learning in Low-rank Orthogonal Subspaces (2020, positive): “Some works [Hayes et al., 2018, Riemer et al., 2019, Rolnick et al., 2018, Chaudhry et al., 2019b, 2020] have shown that directly optimizing the loss on the episodic memory, also known as experience replay, is cheaper than constraint-based approaches and improves prediction performance.”
Linear Mode Connectivity in Multitask and Continual Learning (2020, methodology): “, number of tasks, episodic memory size, and training epochs per task), hyper-parameters, and evaluation metrics are chosen to be similar to several other studies (Chaudhry et al., 2018b; Mirzadeh et al., 2020; Chaudhry et al., 2019; Farajtabar et al., 2019; Chaudhry et al., 2019).”