A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix

By Thang Van Doan, Mehdi Abbana Bennani, Bogdan Mazoure, Guillaume Rabusseau, Pierre Alquier. 2020

120 citations recorded by the source corpus.

Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although major advances have been made in the field, one recurring problem which remains unsolved is that of Catastrophic Forgetting (CF). While the issue has been extensively studied empirically, little attention has been paid from a theoretical angle. In this paper, we show that the impact of CF increases as two tasks increasingly align. We introduce a measure of task similarity called the NTK overlap matrix which is at the core of CF. We analyze common projected gradient algorithms and demonstrate how they mitigate forgetting. Then, we propose a variant of Orthogonal Gradient Descent (OGD) which leverages structure of the data through Principal Component Analysis (PCA). Experiments support our theoretical findings and show how our method reduces CF on classical CL datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Sometheoreticalstudieswiththeneuraltangent kernel (NTK) [96], [97], [98], [99] suggest that an increase in task similarity may lead to more forgetting.”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “We prove Proposition 6 in Appendix C, where we also review the OGD method; see Bennani et al. (2020); Doan et al. (2021) for different theoretical aspects of OGD.”
Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting (2021, methodology): “The trained model within the NTK regime is then given as follows (Bennani et al., 2020; Doan et al., 2021): (5) We omit the index c because each head is updated independently.”
Wide Neural Networks Forget Less Catastrophically (2021, methodology): “In the context of continual learning, (Doan et al., 2021) have studied forgetting of projection-based algorithms in the NTK regime using the similarity of the tasks.”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “Continual learning in the NTK regime Doan et al. (2020) analysed the impact of task similarity, and also found increasing task similarity leads to more forgetting.”
Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation (2020, background): “Doan et al. (2020) presents a theoretical analysis of catastrophic forgetting in the Neural Tangent Kernel regime.”
Regularization Shortcomings for Continual Learning (2019, methodology): “Other methods such as [8,5] are a bit different, they do not add a regularization loss but they regularize the gradient of the loss to protect important weights.”
Efficient Continual Learning Ensembles in Neural Network Subspaces (2022, background): “This setup induces Catastrophic Forgetting (CF) (McCloskey and Cohen, 1989) which is a degradation of performances on previous data due to distribution change between tasks (Doan et al., 2021).”
Provable Lifelong Learning of Representations (2021, methodology): “[Bennani et al., 2020, Doan et al., 2021] analyze the generalization of the OGD algorithm [Farajtabar et al., 2020] through NTK [Jacot et al., 2018].”
Provable and Efficient Continual Representation Learning (2022, background): “We also emphasize that there are several interesting works on the theory of continual learning such as Lee et al. (2021); Doan et al. (2021); Bennani et al. (2020); Mirzadeh et al. (2020); Yin et al. (2020).”
Provable Continual Learning via Sketched Jacobian Approximations (2021, unknown): “Some recent theoretical developments include (Yin et al., 2020; Bennani et al., 2020; Alquier et al., 2017; Doan et al., 2021).”
How catastrophic can catastrophic forgetting be in linear regression? (2022, methodology): “Their work [14] and others [6, 16] also analyzed the OGD algorithm, deriving generalization guarantees and improved variants.”
SketchOGD: Memory-Efficient Continual Learning (2023, unknown): “Another solution called PCA-OGD has been proposed [31], which stores only the top principal components of the gradients for each task.”
Continual Learning in Linear Classification on Separable Data (2023, background): “…there are still very few analytic results on it, since most of the continual learning theory thus far has focused on regression settings (e.g., Bennani et al. (2020); Doan et al. (2021); Asanuma et al. (2021); Lee et al. (2021); Evron et al. (2022); Goldfarb & Hand (2023); Li et al. (2023)).”
Formalizing the Generalization-Forgetting Trade-off in Continual Learning (2021, methodology): “On the other hand, [13] formalized the alignment as an eigenvalue problem and introduced a PCA-driven method to ensure alignment.”
A Survey on Negative Transfer (2020, background): “First, the model may forget earlier knowledge when trained on new tasks, known as catastrophic forgetting [39], [40].”