Learning to Continually Learn

By Shawn Beaulieu, Lapo Frati, Thomas Miconi, Joel Lehman, Kenneth O. Stanley, Jeff Clune, Nick Cheney. 2020

166 citations recorded by the source corpus.

Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves manually-designed solutions to the problem. We instead advocate meta-learning a solution to catastrophic forgetting, allowing AI to learn to continually learn. Inspired by neuromodulatory processes in the brain, we propose A Neuromodulated Meta-Learning Algorithm (ANML). It differentiates through a sequential learning process to meta-learn an activation-gating function that enables context-dependent selective activation within a deep neural network. Specifically, a neuromodulatory (NM) neural network gates the forward pass of another (otherwise normal) neural network called the prediction learning network (PLN). The NM network also thus indirectly controls selective plasticity (i.e. the backward pass of) the PLN. ANML enables continual learning without catastrophic forgetting at scale: it produces state-of-the-art continual learning performance, sequentially learning as many as 600 classes (over 9,000 SGD updates).

Subfields: Education, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “AIM[183]learnsamixtureofexpertstomakepredic-tionswiththerepresentationsofOML[181]orANML[182], furthersparsifyingtherepresentationsatthearchitecturallevel.”
Constrained Few-shot Class-incremental Learning (2022, positive): “As shown in Appendix A.4.3, we find that all modes of C-FSCIL performs consistently better than the best baseline, ANML [2].”
Wide Neural Networks Forget Less Catastrophically (2021, background): “For example, Javed and White (2019); Beaulieu et al. (2020) showed that meta-learned representations forget less when trained continually.”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “Beaulieu et al. [181] proposed a Neuromodulated Meta-Learning Algorithm which contains a neuromodulatory neural network and a prediction learning network.”
Few-Shot and Continual Learning with Attentive Independent Mechanisms (2021, methodology): “The algorithms shown are applicable for both few-shot and continual learning with the distinction between both highlighted with different colors — few-shot learning using SIB in green and continual learning using OML and ANML in blue.”
Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, methodology): “It has been used as a training strategy in sequence learning [43, 44] and continual learning scenarios [45, 46, 47, 48].”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Indeed, this phased learning setting can be adopted for continual learning deployment at meta-test time by training the inner loop over a sequence of tasks (Javed and White, 2019; Spigler, 2019; Beaulieu et al., 2020; Caccia et al., 2020; Kim et al., 2021a; Co-Reyes et al., 2021a).”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, neutral): “, 2020) (Beaulieu et al., 2020)”
Continual Learning in Low-rank Orthogonal Subspaces (2020, positive): “Similarly, Javed and White [2019], Beaulieu et al. [2020] showed that learning transferable representations via meta-learning reduces forgetting when the model is trained on sequential tasks.”
Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “In this sense it connects to topics such as MetaLearning (Beaulieu et al., 2020; Jerfel et al., 2019; He & Jaeger, 2018; Riemer et al., 2018), Few-Shot Learning (Wen et al., 2018; Gidaris & Komodakis, 2018), Multi-task and Transfer Learning (He & Jaeger, 2018; Jerfel et al., 2019).”
Meta-Consolidation for Continual Learning (2020, methodology): “Even meta-learning based continual learning methods that have been proposed in the recent past [34, 24, 6, 64], meta-learn an initialization amenable for quick adaptation across tasks, similar to MAML [23], and hence operate in the weight space.”
Understanding the Role of Training Regimes in Continual Learning (2020, methodology): “Continual Learning as an emerging ﬁeld in Artiﬁcial Intelligence is connected to many other areas such as Meta Learning [4, 33, 22, 59], Few Shot Learning [71, 19], Multi-task and Transfer Learning [22, 33] and the closely related problem of exploring task boundary or task detection [57, 2, 23, 34].”
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer (2021, positive): “Approaches include regularization penalties [50, 51, 52, 53, 54], meta learning [55], model compression [56, 57, 58], or knowledge distillation [5, 7, 8, 9].”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, background): “Despite that, meta-continual learning is actively researched (Riemer et al., 2018; Beaulieu et al., 2020).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “A similar method learns a dual network for gating the outputs of a standard network (Beaulieu et al., 2020).”
MetaFSCIL: A Meta-Learning Approach for Few-Shot Class Incremental Learning (2022, methodology): “MAML has also been adopted to OC-IL [1,11].”
Building a Subspace of Policies for Scalable Continual Learning (2022, background): “There is also a large body of work which leverages the shared structure of the tasks [50, 30, 25, 27, 1, 43], meta-learning [19, 44, 4, 8, 33, 52, 39], or generative models [35, 42, 41, 2] to improve CRL agents, but these are less related to CSP and not very competitive.”