An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

By Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron C. Courville, Yoshua Bengio. 2013

1,666 citations recorded by the source corpus.

Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "forget" how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to which the catastrophic forgetting problem occurs for modern neural networks, comparing both established and recent gradient-based training algorithms and activation functions. We also examine the effect of the relationship between the first task and the second task on catastrophic forgetting. We find that it is always best to train using the dropout algorithm--the dropout algorithm is consistently best at adapting to the new task, remembering the old task, and has the best tradeoff curve between these two extremes. We find that different tasks and relationships between tasks result in very different rankings of activation function performance. This suggests the choice of activation function should always be cross-validated.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

In Optimization & Training

Citations in the corpus

Unsupervised Pretraining for Sequence to Sequence Learning (2016, background): “However, this procedure may lead to catastrophic forgetting, where the model’s performance on the language modeling tasks falls dramatically after fine-tuning (Goodfellow et al., 2013).”
Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “However, continual learning by deep models has proven to be very challenging due to catastrophic forgetting , a long known problem of training deep neural networks [4, 5, 14, 15, 28, 31, 34].”
A Closer Look at Memorization in Deep Networks (2017, methodology): “This is interesting since dropout is also known to prevent catastrophic forgetting (Goodfellow et al., 2013) and thus in general it seems to help DNNs retain patterns.”
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, methodology): “The Permuted MNIST experiment [2] involves ten-digit classification, where each task consists of different permutations of the pixels in the images.”
Learning without Forgetting (2016, background): “Ideally, the new tasks could be learned while sharing parameters from old ones, without suffering from Catastrophic Forgetting [1], [2] (degrading performance on old tasks) or having access to the old training data.”
How To Backdoor Federated Learning (2018, background): “Preventing the backdoor from being forgotten as the model is updated by benign participants is similar to the catastrophic forgetting problem in multi-task learning [22, 33, 39].”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “cquisition (Sec. 4.4) and multisensory systems for crossmodal continual learning (Sec. 4.5). This review complements previous surveys on catastrophic forgetting in connectionist models (French, 1999; Goodfellow et al., 2013; Soltoggio et al., 2017) that do not critically compare recent experimental ”
Continual Learning with Deep Generative Replay (2017, background): “A branch of works assumes a particular situation where access to previous data is limited to the current task[12, 18, 20].”
Three Approaches for Personalization with Applications to Federated Learning (2020, background): “( ) However, to the best of our knowledge, there are no theoretical guarantees and the algorithm may be prone to catastrophic forgetting (Goodfellow et al., 2013).”
Variational Continual Learning (2017, methodology): “Too much plasticity leads to the infamous catastrophic forgetting problem (McCloskey and Cohen, 1989; Ratcliff, 1990; Goodfellow et al., 2014a) and too much stability leads to an inability to adapt.”
Deep learning in neural networks: An overview (2014, background): “In this context, choosing a learning algorithm may be more important than choosing activation functions (Goodfellow et al., 2014a).”
iCaRL: Incremental Classifier and Representation Learning (2016, background): “’s [10] for modern architectures, except in specific settings, such as Kirkpatrick et al.”
Meta Networks (2017, methodology): “Following the previous work on catastrophic forgetting in neural networks (Srivastava et al., 2013; Goodfellow et al., 2014; Kirkpatrick et al., 2016), we formulated two problems in a sequential manner.”
Off-Policy Deep Reinforcement Learning without Exploration (2018, background): “However, even if a sufﬁcient amount of data collection occurs, the concerns of catastrophic forgetting (McCloskey & Cohen, 1989; Goodfellow et al., 2013) may still result in extrapolation error.”
Learning Rapid-Temporal Adaptations (2017, background): “Systems built in such a train-and-then-test manner do not scale to complex, realistic environments and tasks: They require gluts of single task data and are prone to issues related to distributional shifts, such as catastrophic forgetting [37, 8, 16] and adversarial data points [39].”
Overcoming catastrophic forgetting with hard attention to the task (2018, methodology): “We consider 2 reference approaches plus 9 recent and competitive ones: standard SGD with dropout (Goodfellow et al., 2014), SGD freezing all layers except the last one (SGD-F), EWC, incremental moment matching (IMM-Mean and IMM-Mode; Lee et al., 2017), learning without forgetting (LWF; Li & Hoiem,…”
FearNet: Brain-Inspired Model for Incremental Learning (2017, methodology): “FearNet could also be adapted to perform the supervised data permutation experiment performed by Goodfellow et al. (2013) and Kirkpatrick et al. (2017).”