Embracing Change: Continual Learning in Deep Neural Networks.

By Raia Hadsell, Dushyant Rao, Andrei A. Rusu, Razvan Pascanu. 2020

628 citations recorded by the source corpus.

Artificial intelligence research has seen enormous progress over the past few decades, but it predominantly relies on fixed datasets and stationary environments. Continual learning is an increasingly relevant area of study that asks how artificial systems might learn sequentially, as biological systems do, from a continuous stream of correlated data. In the present review, we relate continual learning to the learning dynamics of neural networks, highlighting the potential it has to considerably improve data efficiency. We further consider the many new biologically inspired approaches that have emerged in recent years, focusing on those that utilize regularization, modularity, memory, and meta-learning, and highlight some of the most promising and impactful directions.

Subfields: Artificial Intelligence, Infectious Diseases, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Infectious Diseases

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “In response to external Manuscript received 13 February 2023; revised 13 September 2023; accepted changes, evolution has empowered human and other organisms with strong adaptability to continually acquire, update, accumulate and exploit knowledge [1], [2], [3].”
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, neutral): “Since biological learning systems are natural continual learners that exhibit strong adaptability to real-world dynamics [2, 3, 20], we argue that they have been equipped with effective strategies to address the above challenges.”
Disentangling Transfer in Continual Reinforcement Learning (2022, background): “Continual learning algorithms are often categorized into three classes: regularization-based e.g. [2, 23, 31], parameter isolation e.g. [26] and rehearsal methods e.g. [6, 7]; see also CL survey papers [8, 17, 32].”
Anti-Retroactive Interference for Lifelong Learning (2022, background): “The learning process of new tasks results in catastrophic forgetting [10] of previous knowledge due to retroactive interference [25], which means that the content of later learning competes with the previous memory and interferes with the previous learning.”
NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks (2022, background): “Besides their mathematical foundation, these methods are also motivated by studies of synaptic consolidation (Hadsell et al., 2020).”
Simpler is Better: off-the-shelf Continual Learning Through Pretrained Backbones (2022, background): “For a more complete review on CL methodologies we point out these recent surveys [6,17,19].”
Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, background): “Despite progress in continual learning (Lange et al., 2019; Parisi et al., 2019; Biesialska et al., 2020; Khetarpal et al., 2020a; Hadsell et al., 2020), there is – to the best of our knowledge – no work that addresses continual reinforcement learning (continual RL) of the dialogue policy, even…”
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, background): “Although it is interesting to design various matching and query strategies by introducing additional components, it actually violates the principle of parsimony in continual learning [12, 58].”
Probing Representation Forgetting in Supervised and Unsupervised Continual Learning (2022, background): “Indeed recent works have highlighted the importance of fast remembering versus catastrophic forgetting [14, 17], a looser continual learning requirement where in the task performance may decrease but the agent is able to recover rapidly upon observing a few samples from the previous task.”
Learning to Prompt for Continual Learning (2021, background): “Second, even if the task-independent prompt can be known at test time, it prevents possible knowledge sharing between similar tasks [16].”
Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting (2021, background): “It has attracted much attention due to the demand on applications, and fundamental understanding and algorithms are being explored (Hadsell et al., 2020).”
Continual Learning via Local Module Composition (2021, background): “The rapid growth of continual learning has lead researchers to work on empirical studies [20, 58, 56], surveys [32, 42, 55, 66, 67] as well as CL-specific software [72, 22, 59].”
Kernel Continual Learning (2021, background): “While alleviating forgetting, the penalty term also unavoidably prevents the plasticity to absorb new information from future tasks learned over a long timescale (Hadsell et al., 2020).”
Learngene: From Open-World to Your Learning Task (2021, background): “There are currently some learning paradigms that can solve some of the above problems, such as continual learning (Hadsell et al. 2020), transfer learning (Perkins, Salomon et al.”
Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021, background): “[18] provide a high-level overview of CL and argue that learning in a non-stationary setting is a fundamental problem for the development of AI, highlighting the frequent connections to neuroscience.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “, 2020) and (Hadsell et al., 2020), Most of this work considers the task incremental learning setting.”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “…to design models that continually accumulate new knowledge without forgetting relevant information about the past (Mccloskey and Cohen, 1989; Thrun and Mitchell, 1995; French, 1999; Mitchell et al., 2015; Rusu et al., 2016; Kirkpatrick et al., 2017; Al-Shedivat et al., 2018; Hadsell et al., 2020).”