A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning

By Martin Mundt, Yongjun Hong, Iuliia Pliushch, Visvanathan Ramesh. 2020

183 citations recorded by the source corpus.

Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.

Subfields: Artificial Intelligence, Computational Mechanics, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Meta & Continual Learning

Citations in the corpus

Few-Shot Continual Active Learning by a Robot (2022, background): “[17] present a thorough analysis of previous continual and active learning, and OOD works, and suggest a common viewpoint with open-set recognition acting as a natural interface between continual and active learning.”
Continual Learning via Local Module Composition (2021, background): “The rapid growth of continual learning has lead researchers to work on empirical studies [20, 58, 56], surveys [32, 42, 55, 66, 67] as well as CL-specific software [72, 22, 59].”
Posterior Meta-Replay for Continual Learning (2021, neutral): “[51] M.”
Continual Learning in Task-Oriented Dialogue Systems (2020, background): “Interested readers may refer to Mundt et al. (2020); Parisi et al. (2019); De Lange et al. (2019) for an overview of existing approaches, and to Section 2.2 for more details on the three main CL approaches studied in this paper.”
Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer (2021, neutral): “We acknowledge other concurrent works push realistic continual learning settings as well (Mundt et al., 2020; Ren et al., 2020), but the contributions are orthogonal to our setting.”
CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability (2021, methodology): “Correspondingly, this utility measure can either require presence of supervision, as shown in the class-specific meta-recognition model of OCDVAE, or can be entirely unsupervised.”
Pretrained Language Model in Continual Learning: A Comparative Study (2022, background): “, 2021b), surveys (Hadsell et al., 2020; Khetarpal et al., 2020; Lesort et al., 2021a; Mundt et al., 2020; 2021) as well as CL-specific software (Normandin et al.”
Opening up Open World Tracking (2021, background): “Open-world recognition models must explicitly specify unknown objects [7, 58, 70].”
Scaling the Number of Tasks in Continual Learning (2022, methodology): “• We propose a new experimentation framework SCoLe based on the ﬁnite world assumption Mundt et al. [2020] with a potentially inﬁnite number of tasks.”
Energy-Based Models for Continual Learning (2020, background): “Although this might seem straightforward, critical non-trivial questions are how to select the data to be stored as well as exactly how to use them (Rebuffi et al., 2017; Lopez-Paz & Ranzato, 2017; Rajasegaran et al., 2019; Hou et al., 2019; Wu et al., 2019; Mundt et al., 2020).”
Understanding Continual Learning Settings with Data Distribution Drift Analysis (2021, background): “This assumption is denoted “closed world assumption” (Mundt et al., 2020).”
Class-Incremental Learning with Generative Classifiers (2021, background): “Moreover, there are also other tricks that could speed up inference: it might be possible to use uncertainty estimates (which can be obtained from the generative models [37]) to inform the number of importance samples to use, or the classification decision could be made hierarchical (e.”
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents (2021, methodology): “For a overview on continual learning applied to neural networks in general, we refer the reader to [49, 45].”
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, methodology): “The main idea of replay-based methods is to re-train samples or representations from already seen tasks when learning new tasks (Mundt et al., 2020).”
One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning (2023, background): “These strategies are detailed in several comprehensive surveys (De Lange et al., 2021; Mai et al., 2022; Mundt et al., 2023).”