Continual Lifelong Learning with Neural Networks: A Review

By German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, Stefan Wermter. Neural Networks, 2018

3,616 citations recorded by the source corpus.

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor skills as well as to long-term memory consolidation and retrieval. Consequently, lifelong learning capabilities are crucial for computational learning systems and autonomous agents interacting in the real world and processing continuous streams of information. However, lifelong learning remains a long-standing challenge for machine learning and neural network models since the continual acquisition of incrementally available information from non-stationary data distributions generally leads to catastrophic forgetting or interference. This limitation represents a major drawback for state-of-the-art deep neural network models that typically learn representations from stationary batches of training data, thus without accounting for situations in which information becomes incrementally available over time. In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. Although significant advances have been made in domain-specific learning with neural networks, extensive research efforts are required for the development of robust lifelong learning on autonomous agents and robots. We discuss well-established and emerging research motivated by lifelong learning factors in biological systems such as structural plasticity, memory replay, curriculum and transfer learning, intrinsic motivation, and multisensory integration.

Subfields: Representation & Self-Supervised Learning

View Paper: https://arxiv.org/abs/1802.07569

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “…learning (Lake et al., 2011; Romera-Paredes and Torr, 2015; Vinyals et al., 2016; Pan and Yang, 2010), meta-learning and life-long learning (Thrun and Mitchell, 1995; Silver et al., 2013; Chen and Liu, 2016; Parisi et al., 2018; Golkar et al., 2019; Sodhani et al., 2018; Lopez-Paz et al., 2017).”
Toward Understanding Catastrophic Forgetting in Continual Learning (2019, background): “For more comprehensive reviews of continual learning, we refer to Chen et al. [11] and Parisi et al. [30]”
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines (2018, background): “Such single-headed Split MNIST is known as incremental class learning [11, 12, 13].”
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs (2021, background): “This setting bears some commonality to the continual learning problem, though continual learning methods generally aim to learn new tasks or datasets rather than make targeted updates to specific model beliefs (Parisi et al., 2019).”
Fast Model Editing at Scale (2021, background): “However, in continual learning settings, a model is typically expected to learn wholly new behaviors or datasets (Kirkpatrick et al., 2017; Parisi et al., 2019) without forgetting, while in this work we consider more localized model edits.”
Experience Replay for Continual Learning (2018, background): “In recent years, however, there has been renewed interest in overcoming catastrophic forgetting in RL and in supervised learning from streaming data, where it is not possible simply to reorder the data (see e.g. [8, 17, 24, 26]).”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Although there are some early surveys of continual learning with relatively broad coverage [2], [5], [8], important advances in recent years have not been incorporated.”
Weight Agnostic Neural Networks (2019, background): “The ability to quickly ﬁne-tune weights is useful in few-shot learning [21] and may ﬁnd uses in continual lifelong learning where agents continually acquire, ﬁne-tune, and transfer skills throughout their lifespan [83].”
A Survey on Multi-Task Learning (2017, background): “Continual learning [4], in which tasks come sequentially, learns tasks one by one,”
On the role of neurogenesis in overcoming catastrophic forgetting (2018, background): “Since real-world data is naturally non-stationary - for example, video may focus on different scenes or objects in adjacent temporal blocks - it is desirable for learning algorithms to operate on non-stationary input distributions without suffering from catastrophic forgetting [2], i.”
Continuous Learning in Single-Incremental-Task Scenarios (2018, methodology): “During the past few years we have witnessed a renewed and growing attention to Continuous Learning (CL) [3][24].”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “, 1998) and CIFAR-10 (Krizhevsky, 2009) while not scaling up to incremental learning tasks with larger-scale datasets of natural images and videos (Kemker et al., 2018; Parisi et al., 2018a).”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “An important open challenge for machine learning in general is the creation of neural systems that can continuously integrate new knowledge and skills without forgetting what they previously learned (Parisi et al., 2018), thus solving the stability-plasticity dilemma.”
Closed-Loop GAN for continual Learning (2018, neutral): “Regularization methods have been shown to perform poorly in incremental class learning [Kemker and Kanan, 2017, Parisi et al., 2018] and here we reproduce this limitation in our own results for the case of Elastic Weight Consolidation [Kirkpatrick et al., 2017].”
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, neutral): “As biological learning systems are natural continual learners that show strong adaptability to real-world changes 2,3,18 , we argue that they have been equipped with effective strategies to address the above challenges.”
The Ideal Continual Learner: An Agent That Never Forgets (2023, neutral): “…Ramesh & Chaudhari, 2022; Douillard et al., 2022), or combinations thereof (Yan et al., 2021; Xie et al., 2022; Wang et al., 2022b; Frascaroli et al., 2023); for surveys, see, e.g., Parisi et al. (2019); De Lange et al. (2022); Qu et al. (2021); Shaheen et al. (2022); van de Ven et al. (2022).”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, background): “Rehearsal-based Approaches: Continual learning over a sequence of tasks has been a longstanding challenge, since learning a new task causes large weight changes in the DNNs, resulting in overfitting on the current task and catastrophic forgetting of older tasks (Parisi et al., 2019).”
FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning (2022, background): “CIL algorithms are needed when data arrive sequentially and/or computational constraints are important [9, 17, 24, 29].”