Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

By Nicolas Y. Masse, Gregory D. Grant, David J. Freedman. 2018

322 citations recorded by the source corpus.

Significance Artificial neural networks can suffer from catastrophic forgetting, in which learning a new task causes the network to forget how to perform previous tasks. While previous studies have proposed various methods that can alleviate forgetting over small numbers (⩽10) of tasks, it is uncertain whether they can prevent forgetting across larger numbers of tasks. In this study, we propose a neuroscience-inspired scheme, called “context-dependent gating,” in which mostly nonoverlapping sets of units are active for any one task. Importantly, context-dependent gating has a straightforward implementation, requires little extra computational overhead, and when combined with previous methods to stabilize connection weights, can allow networks to maintain high performance across large numbers of sequentially presented tasks. Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of “catastrophic forgetting,” in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly nonoverlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that using multiple, complementary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

In Meta & Continual Learning

Citations in the corpus

Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning (2019, background): “However, if erroneous contextual information is learned in previous layers, it is difﬁcult to correct (Masse, Grant, and Freedman 2018), which is a problem magniﬁed by noisy data and content sparsity (e.g. Twitter, Reddit, Blogs).”
Balancing Stability and Plasticity through Advanced Null Space in Continual Learning (2022, methodology): “Architecture-based methods allocate diﬀerent parameters or add new parameters for the new task, while sharing parameters across tasks to reduce the interference of previous tasks [3, 22, 27–29, 41, 42, 44, 53, 56, 60].”
Incremental Task Learning with Incremental Rank Updates (2022, background): “[20] proposes nonoverlapping sets of units being active for each task.”
Continual Learning with Guarantees via Weight Interval Constraints (2022, methodology): “…et al., 2017; Lee et al., 2017; Li & Hoiem, 2017; Lopez-Paz & Ranzato, 2017; Shin et al., 2017; Zenke et al., 2017; Aljundi et al., 2018; Masse et al., 2018; Liu et al., 2018; Rolnick et al., 2019; van de Ven et al., 2020), they typically do not provide any firm guarantees about the…”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “In learning a new task, the learner has to update the network parameters but this update can cause the models for previous tasks to degrade or to be forgotten (McCloskey and Cohen, 1989).”
Kernel Continual Learning (2021, background): “This is achieved by a gating mechanism (Wortsman et al., 2020; Masse et al., 2018), or by incrementally adding new parameters to the model (Rusu et al., 2016).”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, background): “In the case of artificial neural networks (ANNs), task interference is behind the problem known as Catastrophic Forgetting (CF) [3, 4, 5, 6, 7].”
Do not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting (2021, methodology): “The task-specific methods deal with adding a separate model or layer for each new task [28, 25].”
Distill and Replay for Continual Language Learning (2020, background): “The other stretch is model-based that enables continual learning with modification to the model architecture (Schwarz et al., 2018; Masse et al., 2018).”
Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “Masse et al. (2018) proposes a bio-inspired approach for a context-dependent gating that activates non-overlapping subset of parameters for any specific task.”
Supermasks in Superposition (2020, methodology): “More efficiently, [32] fixes the network size and randomly assigns which nodes are active for a given task.”
GAN Memory with No Forgetting (2020, neutral): “, regularization-based [36, 69, 95, 55, 68], dynamic-model-based [47, 68, 47], and generativereplay-based methods [72, 42, 86, 66, 88, 78].”
Understanding the Role of Training Regimes in Continual Learning (2020, methodology): “[50] proposes a neuroscience-inspired method for a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task.”
Dropout as an Implicit Gating Mechanism For Continual Learning (2020, methodology): “In the comprehensive set of experiments, PackNet is shown one of the most reliable methods [13] and adding the context-dependent-gates to other methods such as EWC improved their performance drastically [17].”
Learning to Continually Learn (2020, methodology): “eta-training dataset. 5 Sequences up to 500 have been tried, but only when providing a knowledge of which unique task is being solved directly to the network instead of requiring it to learn that too [32]. Figure2 : Meta-testtraining classiﬁcation accuracy. The x-axis shows the number of sequential ”
Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control (2019, background): “…architectures for neural networks,(Fernando et al. 2017; Aljundi, Chakravarty, and Tuytelaars 2017; Lee et al. 2017a; Serrà et al. 2018; Masse, Grant, and Freedman 2018; Siavash Golkar 2019) where the capacity to learn the new task is obtained by assigning different parts of the…”
Continual learning with hypernetworks (2019, background): “This interpretation is consistent with a growing body of work which suggests the involvement of modulatory inputs in implementing context-or task-dependent network mode-switching (Mante et al., 2013; Jaeger, 2014; Stroud et al., 2018; Masse et al., 2018).”