Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks

By Roby Velez, Jeff Clune. 2017

52 citations recorded by the source corpus.

A long-term goal of AI is to produce agents that can learn a diversity of skills throughout their lifetimes and continuously improve those skills via experience. A longstanding obstacle towards that goal is catastrophic forgetting, which is when learning new information erases previously learned information. Catastrophic forgetting occurs in artificial neural networks (ANNs), which have fueled most recent advances in AI. A recent paper proposed that catastrophic forgetting in ANNs can be reduced by promoting modularity, which can limit forgetting by isolating task information to specific clusters of nodes and connections (functional modules). While the prior work did show that modular ANNs suffered less from catastrophic forgetting, it was not able to produce ANNs that possessed task-specific functional modules, thereby leaving the main theory regarding modularity and forgetting untested. We introduce diffusion-based neuromodulation, which simulates the release of diffusing, neuromodulatory chemicals within an ANN that can modulate (i.e. up or down regulate) learning in a spatial region. On the simple diagnostic problem from the prior work, diffusion-based neuromodulation 1) induces task-specific learning in groups of nodes and connections (task-specific localized learning), which 2) produces functional modules for each subtask, and 3) yields higher performance by eliminating catastrophic forgetting. Overall, our results suggest that diffusion-based neuromodulation promotes task-specific localized learning and functional modularity, which can help solve the challenging, but important problem of catastrophic forgetting.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Meta & Continual Learning

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “dulation produced forward models that could adapt to changes signiﬁcantly better than the controls. They veriﬁed that evolution exploited variable learning rates to perform adaptation when needed. In Velez and Clune (2017), diffusionbased modulation, i.e., targeting entire parts of the network, evol”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, background): “…and Jaeger, 2018), Gating Networks (Masse et al., 2018; Serrà et al., 2018), PackNet (Mallya and Lazebnik, 2018), Diffusion-based Neuromodulation (Velez and Clune, 2017), IMM (Lee et al., 2017), Expandable Networks (Yoon et al., 2018; Li et al., 2019), RPSNet (Rajasegaran et al., 2019),…”
Learning to Continually Learn (2020, background): “There is prior work in machine learning that harnesses neuromodulation techniques [7, 16, 39, 40, 42], but in such work neuromodulation directly modulates learning rates, instead of the approach taken in this work of directly modulating activations and thus indirectly controlling learning.”
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization (2018, background): “Strategies that encourage the development of a modular representation, as is believed to occur in the brain (25), might be required to fully implement continual and transfer learning (26).”
Go-Explore: a New Approach for Hard-Exploration Problems (2019, background): “It likely would also have no implicit memory of this frontier due to the problem of catastrophic forgetting [17–20].”
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity (2018, background): “By activating plasticity only in the subset of neural weights relevant for the task currently being performed, knowledge stored in other weights about different tasks is left untouched, alleviating catastrophic forgetting (Ellefsen et al., 2015; Velez & Clune, 2017).”
Biological underpinnings for lifelong learning machines (2022, background): “The role of neuromodulation in machine learning systems has been extensively explored 79,84–86, 88, 89, Context-dependent perception and gating.”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, methodology): “Additionally, there is a recent increase in meta-learning the learning algorithms themselves [37, 191, 40, 187, 161] (discussed more in Section 2.2).”