Toward Understanding Catastrophic Forgetting in Continual Learning

By Cuong V Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, Stefano Soatto. 2019

124 citations recorded by the source corpus.

We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure that makes use of recent developments in task space modeling as well as correlation analysis to specify and analyze the properties we are interested in. As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity. We show that error rates are strongly and positively correlated to a task sequence's total complexity for some state-of-the-art algorithms. We also show that, surprisingly, the error rates have no or even negative correlations in some cases to sequential heterogeneity. Our findings suggest directions for improving continual learning benchmarks and methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

Citations in the corpus

CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, methodology): “We could instead update the model by only training on the new training data, but this leads to a phenomenon known as catastrophic forgetting [46] where the model overwrites existing knowledge when learning the new data, a problem known as continual learning.”
Probing Representation Forgetting in Supervised and Unsupervised Continual Learning (2022, background): “Recent works on elucidating the nature of catastrophic forgetting have examined the influence of task sequence [33], network architecture [3], and change in representation similarity [38].”
Wide Neural Networks Forget Less Catastrophically (2021, background): “Alleviating catastrophic forgetting has attracted a lot of attention lately, and many interesting solutions are proposed to partly overcome the issue (e.g., Toneva et al., 2019; Nguyen et al., 2019; Hsu et al., 2018; Li et al., 2019; Wallingford et al., 2020).”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “Similarly, a series of works has started a systematic empirical analysis of this phenomenon in deep networks (Parisi et al., 2019; Mirzadeh et al., 2020; Neyshabur et al., 2020; Nguyen et al., 2019; Ruder & Plank, 2017).”
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix (2020, background): “Previous work either investigated this phenomenon empirically at diﬀerent granularity levels (task level [Nguyen et al., 2019], neural network representations level [Ramasesh et al., 2020] ), or proposed a quantitative metric [Farquhar and Gal, 2018, Kemker et al., 2017, Nguyen et al., 2020].”
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics (2020, background): “Otherprior work on understanding catastrophic forgetting has looked at properties of task sequences [33] or as a means of comparing diﬀerent neural network architectures [34].”
On Class Orderings for Incremental Learning (2020, methodology): “, 2019), or the one proposed in (Nguyen et al., 2019) for IL.”
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent (2020, methodology): “Several heuristics were developed in order to characterise it (Ans & Rousset 1997, Ans & Rousset 2000, Goodfellow et al. 2014, French 1999, McCloskey & Cohen 1989, Robins 1995, Nguyen et al. 2019).”
Understanding the Role of Training Regimes in Continual Learning (2020, background): “A growing body of research has attempted to tackle this problem in recent years [58, 72, 57, 29].”
Orthogonal Gradient Descent for Continual Learning (2019, background): “, 2018), and understating this phenomenon (Nguyen et al., 2019; Farquhar and Gal, 2019).”
Transferability and Hardness of Supervised Classification Tasks (2019, background): “(7) using an existing correlation analysis [43].”
Continual Learning with Foundation Models: An Empirical Study of Latent Replay (2022, background): “Several recent works have investigated the role of task similarity for catastrophic forgetting (Nguyen et al., 2019; Ramasesh et al., 2021; Lee et al., 2021).”
A Review of Deep Transfer Learning and Recent Advancements (2022, methodology): “Catastrophic forgetting happens because the weights throughout the model can drastically change in the process of finetuning on target data [2, 4].”
A Continual Learning Survey: Defying Forgetting in Classification Tasks (2019, result): “Instead, experimental results on both datasets exhibit ordering agnostic behaviour, corresponding to similar previous ﬁndings [96].”
Explaining How Deep Neural Networks Forget by Deep Visualization (2020, unknown): “[10] study which properties cause the hardness for the learning process.”
Learn from Others and Be Yourself in Heterogeneous Federated Learning (2022, background): “The challenge lies in the continuously changing class distributions of each task [63, 81].”
Does an LSTM forget more than a CNN? An empirical study of catastrophic forgetting in NLP (2019, methodology): “Both earlier empirical studies focused on only two-task sequences, whereas we use four-task sequences, based on Nguyen et al. (2019) who studied the effect of total sequence complexity and sequential heterogeneity.”