Posterior Meta-Replay for Continual Learning

By Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, João Sacramento, Benjamin F. Grewe. 2021

76 citations recorded by the source corpus.

Continual Learning (CL) algorithms have recently received a lot of attention as they attempt to overcome the need to train with an i.i.d. sample from some unknown target data distribution. Building on prior work, we study principled ways to tackle the CL problem by adopting a Bayesian perspective and focus on continually learning a task-specific posterior distribution via a shared meta-model, a task-conditioned hypernetwork. This approach, which we term Posterior-replay CL, is in sharp contrast to most Bayesian CL approaches that focus on the recursive update of a single posterior distribution. The benefits of our approach are (1) an increased flexibility to model solutions in weight space and therewith less susceptibility to task dissimilarity, (2) access to principled task-specific predictive uncertainty estimates, that can be used to infer task identity during test time and to detect task boundaries during training, and (3) the ability to revisit and update task-specific posteriors in a principled manner without requiring access to past data. The proposed framework is versatile, which we demonstrate using simple posterior approximations (such as Gaussians) as well as powerful, implicit distributions modelled via a neural network. We illustrate the conceptual advance of our framework on low-dimensional problems and show performance gains on computer vision benchmarks.

Subfields: Artificial Intelligence, Computational Mechanics, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “This is closely related to the out-of-distribution (OOD) detection, where the predictive uncertainty should be low for in-distribution data and high for OOD data [82], [160], [217].”
A Theoretical Study on Solving Continual Learning (2022, methodology): “Finally, for parameter isolation, we use CCG [63], HyperNet [55], HAT [3], SupSup [4] (Sup), and PR [65].”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “As a result, replay has become a very successful approach for tackling continual RL (Isele and Cosgun, 2018; Riemer et al., 2019; Rolnick et al., 2019; Oh et al., 2021; Henning et al., 2021; Liu et al., 2021b; Queeney et al., 2021; Chandak et al., 2021; Lampinen et al., 2021; Venuto et al., 2021; Li”
Energy-Based Models for Continual Learning (2020, methodology): “Several recent methods (Wortsman et al., 2020; Henning et al., 2021; Yan et al., 2021; Douillard et al., 2022) also achieve good performance on Class-IL without using replay.”
A Unified and General Framework for Continual Learning (2024, unknown): “Representative works include (Nguyen et al., 2018; Kurle et al., 2019; Kao et al., 2021; Henning et al., 2021; Pan et al., 2020; Titsias et al., 2020; Rudner et al., 2022).”
Three types of incremental learning (2022, unknown): “Because all output units should always be active for the Bayesian interpretation of the parameter regularization methods 21 , we decided to use that approach in this study.”
A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, methodology): “Our method enables each task model to produce better scores on OOD outputs and should also be applicable to HyperNet and PMCL. SupSup (Wortsman et al., 2020) predicts task-id by finding the optimal superpositions at inference.”
Open-World Continual Learning: Unifying Novelty Detection and Continual Learning (2023, methodology): “HyperNet [45] and PR [22] propose an entropy based task-id prediction method.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Other works have learned hypernetworks that take a task descriptor as input and output the parameters for a task-specific network, using replay at the task-descriptor level (von Oswald et al., 2020; Henning et al., 2021).”