Functional Regularisation for Continual Learning using Gaussian Processes

By Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh. 2019

217 citations recorded by the source corpus.

We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constructs posterior beliefs over the task-specific functions by using inducing point sparse Gaussian process methods. At each step a new task is first learnt and then a summary is constructed consisting of (i) inducing inputs -- a fixed-size subset of the task inputs selected such that it optimally represents the task -- and (ii) a posterior distribution over the function values at these inputs. This summary then regularises learning of future tasks, through Kullback-Leibler regularisation terms. Our method thus unites approaches focused on (pseudo-)rehearsal with those derived from a sequential Bayesian inference perspective in a principled way, leading to strong results on accepted benchmarks.

Subfields: Probabilistic & Bayesian Methods, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Meanwhile, sequential Bayesian inference over function space can be seen as a form of function regularization, which generally requires storing some old training samples (called “coreset” in literature), such as FRCL [127], FROMP [128] and S-FSVI [60].”
Continual Learning via Sequential Function-Space Variational Inference (2023, methodology): ““Functional regularization for continual learning” ( FRCL ; Titsias et al., 2020) considers a network whose final layer is a Bayesian linear model.”
Information-theoretic Online Memory Selection for Continual Learning (2022, neutral): “However, the majority of approaches focus on task-based continual learning and update the memory relying on task boundaries (Rebuffi et al., 2017; Nguyen et al., 2018; Titsias et al., 2020; Pan et al., 2020; Bang et al., 2021; Yoon et al., 2021).”
The Challenges of Continuous Self-Supervised Learning (2022, background): “In both cases, the model is expected to learn distinct tasks presented sequentially, without forgetting previous ones [32, 38, 64, 76, 85].”
Wide Neural Networks Forget Less Catastrophically (2021, background): “While regularization-based methods provide strong results for a small number of tasks, the importance of parameters drifts when the number of tasks is large (Chaudhry et al., 2018; Titsias et al., 2019).”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “But, unlike [191] which requires solving a discrete optimisation problem to select inducing points, they proposed to store data instances that are close to the decision boundary.”
Kernel Continual Learning (2021, methodology): “Naturally, these methods can also be explored from the perspective of Bayesian optimization (Nguyen et al., 2018; Titsias et al., 2020; Schwarz et al., 2018; Ebrahimi et al., 2020; Ritter et al., 2018).”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, background): “, penalize changes to model parameters) [1, 11, 28, 53, 61] or the prediction space (i.”
Natural continual learning: success is a journey, not (just) a destination (2021, methodology): “While we enforce this stringent condition in this paper, our approach may be easily combined with memory-based techniques such as coresets or generative replay [8, 10, 13, 31, 33, 35, 37, 41, 42, 44, 47].”
Same State, Different Task: Continual Reinforcement Learning without Interference (2021, background): “Alternatively, previous task functions can be regularized to ensure that the functions mapping inputs to outputs are remembered [38, 60, 49].”
Online Coreset Selection for Rehearsal-based Continual Learning (2021, positive): “To tackle this problem, rehearsal-based methods [31, 9, 41] update the model on a randomly sampled replay buffer Ck constructed from the previously observed tasks, where Ck = {xk,j , yk,j} j=1 ∼ Dk, ∀k < t and Jk Nk.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “…and Wermter, 2018; He, Wang, Shan, & Chen, 2018; Wu et al., 2018; Ostapenko et al., 2019; Abati et al., 2020; Liu, Su, Liu, Schiele, & Sun, 2020; Titsias, Schwarz, de G. Matthews, Pascanu, & Teh, 2020; von Oswald, Henning, Sacramento, & Grewe, 2020; Ye & Bors, 2020) or midlevel CNN feature…”
GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “Notable studies that use a similar procedure to ours are Titsias et al. (2019) which regularize subsequent tasks with a set of inducing points stored from previous tasks, and Gidaris & Komodakis (2018) which also freeze the feature extractor after learning the base classes and infer a…”
Continual Learning in Low-rank Orthogonal Subspaces (2020, critical): “For the large number of tasks, the parameter importance measures suffer from brittleness as the locality assumption embedded in the regularization-based approaches is violated [Titsias et al., 2019].”
Meta-Consolidation for Continual Learning (2020, background): “One line of work [88, 38, 47, 2, 16, 71, 56, 13, 79, 42] constrains the parameters of the deep network trained on Task A to not change much while learning a new Task B, while another - replay-based methods - store [63, 48, 14, 66, 9, 3, 4, 22] or generate [74, 81, 46, 45] examples of previous tasks ”