Task-Agnostic Continual Learning Using Online Variational Bayes With Fixed-Point Updates

By Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry. 2020

51 citations recorded by the source corpus.

Abstract Catastrophic forgetting is the notorious vulnerability of neural networks to the changes in the data distribution during learning. This phenomenon has long been considered a major obstacle for using learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, only a few works consider scenarios in which task boundaries are unknown or not well defined: task-agnostic scenarios. The optimal Bayesian solution for this requires an intractable online Bayes update to the weights posterior. We aim to approximate the online Bayes update as accurately as possible. To do so, we derive novel fixed-point equations for the online variational Bayes optimization problem for multivariate gaussian parametric distributions. By iterating the posterior through these fixed-point equations, we obtain an algorithm (FOO-VB) for continual learning that can handle nonstationary data distribution using a fixed architecture and without using external memory (i.e., without access to previous data). We demonstrate that our method (FOO-VB) outperforms existing methods in task-agnostic scenarios. FOO-VB Pytorch implementation is available at https://github.com/chenzeno/FOO-VB.

Subfields: Probabilistic & Bayesian Methods, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Meta & Continual Learning

Citations in the corpus

The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “…and more challenging case of task-agnostic or task-free continual learning, where in the training phase the learner has no access to task identities (Zeno et al., 2018; 2021; Aljundi et al., 2019a; Lee et al., 2020; Jin et al., 2021; Wang et al., 2022c; Pourcel et al., 2022; Ye & Bors, 2022).”
Posterior Meta-Replay for Continual Learning (2021, methodology): “In this context, it is important to note that unlike non-Bayesian CL methods, our approach can utilize any type of online prior-focused method (such as FOO-VB [97]) to also learn within tasks in a non-i.”
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams (2020, neutral): “The data incremental learning paradigm is also referred to as task-free [3] or task-agnostic [54] learning, but is here defined based on transition t of the horizon, which is more general than the task-based definition generating possible ambiguity of how a task is specifically defined.”
Continual Learning with Adaptive Weights (CLAW) (2019, background): “…Ratcliff, 1990; Robins, 1993; 1995; French, 1999; Pape et al., 2011; Goodfellow et al., 2014a; Achille et al., 2018; Kemker et al., 2018; Kemker & Kanan, 2018; Diaz-Rodriguez et al., 2018; Zeno et al., 2018; Ahn et al., 2019; Parisi et al., 2019; Pfulb & Gepperth, 2019; Rajasegaran et al., 2019).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “One early method uses a diagonal Gaussian approximation in order to obtain a closed-form update rule based on the variational free energy (Zeno et al., 2018).”
Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning (2022, background): “…et al., 2016; Fernando et al., 2017; Mallya & Lazebnik, 2018; Xu & Zhu, 2018; Maltoni & Lomonaco, 2019), regularization strategies for consolidating parameters (Kirkpatrick et al., 2017; Zenke et al., 2017), and correcting for domain shifts (Li & Hoiem, 2017; Lee et al., 2017; Zeno et al., 2021).”
Omnidirectional Transfer for Quasilinear Lifelong Learning (2021, background): “In ‘task-unaware’ (or task agnostic [26]) scenarios the learner may not know that the task has changed at all, which means that the hypotheses are of the form, h : X → Y .”
Energy-Based Models for Continual Learning (2020, methodology): “We use the code of “continuous task-agnostic learning” proposed by (Zeno et al., 2018) to generate the training data stream.”
How catastrophic can catastrophic forgetting be in linear regression? (2022, background): “[79]), or in the function space, to protect the outputs of previous tasks (e.”