Overcoming catastrophic forgetting in neural networks
By James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska and 4 more. 2016
10,384 citations recorded by the source corpus.
Significance Deep neural networks are currently the most successful machine-learning technique for solving a variety of tasks, including language translation, image classification, and image generation. One weakness of such models is that, unlike humans, they are unable to learn multiple tasks sequentially. In this work we propose a practical solution to train such models sequentially by protecting the weights important for previous tasks. This approach, inspired by synaptic consolidation in neuroscience, enables state of the art results on multiple reinforcement learning problems experienced sequentially. The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yoshua Bengio
- Sergey Levine
- Jianfeng Gao
- Percy Liang
- Quoc V. Le
- Luke S. Zettlemoyer
- Dacheng Tao
In Statistical and Nonlinear Physics
- Yoshua Bengio
- Fei Ye
- Adrian G. Bors
- Sergey Levine
- Stefano Ermon
- Aaron C. Courville
- Jun Zhu
- Razvan Pascanu
In Optimization & Training
- Yoshua Bengio
- Sergey Levine
- Razvan Pascanu
- Stefano Soatto
- Martin Jaggi
- Peter Richtárik
- Quoc V. Le
- Simon Lacoste-Julien
Citations in the corpus
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, background): “For example, RLHF V3 struggled more than previous versions to compose rhyming lines in poems, as discerned through qualitative analysis, suggesting that further investigation into the causes of and mitigations for forgetting (Kirkpatrick et al., 2017; Nguyen et al., 2019; Ramasesh et al., 2021) coul”
- Learning and Evaluating General Linguistic Intelligence (2019, methodology): “For instance, techniques such as elastic weight consolidation (Kirkpatrick et al., 2017) and progress and compress (Schwarz et al., 2018) may hold promise for improving our models’ robustness to forgetting.”
- The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “, 2017) by penalizing the norm of parameters when training on a new task (Kirkpatrick et al., 2017), the norm of the difference between parameters for previously learned tasks during parameter updates (Hashimoto et al.”
- Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “, 2017) without forgetting existing tasks or languages (Rusu et al., 2016; Kirkpatrick et al., 2017; Lakew et al., 2018).”
- Transfer Learning in Natural Language Processing (2019, background): “…a varying portion of the pre-trained model (Long et al., 2015; Felbo et al., 2017; Howard and Ruder, 2018) with specifically designed regularization (Wiese et al., 2017; Kirkpatrick et al., 2017) or even fine-tuning in sequence a model on a series of datasets using several training objectives.”
- Parameter-Efficient Transfer Learning for NLP (2019, background): “Techniques have been proposed to mitigate forgetting (Kirk- patrick et al., 2017; Zenke et al., 2017), however, unlike for adapters, the memory is still imperfect.”
- On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “Even under randomly shuffled data order, neural networks are known to undergo changes in predictive behavior during training [27, 30, 51, 52].”
- The Curse of Recursion: Training on Generated Data Makes Models Forget (2023, background): “A typical challenge in continual learning is that the model forgets previous samples when learning new information; this is known as catastrophic forgetting [Kirkpatrick et al., 2017].”
- Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, background): “, 2010) and avoiding catastrophic forgetting (French, 1999; Kirkpatrick et al., 2017) have a particular flavor in the domain of conversation: the general task is always to talk to people about any subject, but the set of people, the topics of interest, the facts that have happened may all change.”
- Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “In recent years, interest in continual learning has risen [1, 21, 26, 27, 29, 39, 40, 42, 51], especially from the deep learning research community, due to its potential to reduce training time and training set sizes (e.g., by continuously adapting from previous models), both of which are critical…”
- Generative Adversarial Networks (2021, neutral): “Neural networks have been shown to forget previous learned tasks [17, 35], and catastrophic forgetting was previously considered as a major cause for GAN training instability.”
- Self-Supervised GANs via Auxiliary Rotation Loss (2018, methodology): “The main remedy suggested in the literature is to introduce Method Accuracy Context [24] 0.317 BiGAN [34, 36] 0.310 Colorization [25] 0.326 RotNet [26] 0.387 DeepClustering [35] [15] counter discriminator forgetting by leveraging techniques from continual learning directly (Elastic Weight Sharing [1”
- Injecting Numerical Reasoning Skills into Language Models (2020, methodology): “Pre-training is done in a multi-task setup with a standard LM objective, in order to avoid “catastrophic forgetting” (Kirkpatrick et al., 2017) of the linguistic information in the original LM.”
- LoRA: Low-Rank Adaptation of Large Language Models (2021, background): “Specifically, preforming the usual weight decay on A and B is similar to decaying back to the pre-trained weights, which has been studied as a potentially effective form of regularization against “catastrophic forgetting” [16, 18].”
- Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, methodology): “In other work this is only evaluated in one specific scenario (permuted MNIST) with limited length (3) of task sequence [3].”