Uncertainty-guided Continual Learning with Bayesian Neural Networks

By Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, Marcus Rohrbach. 2019

214 citations recorded by the source corpus.

Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation to measure the parameters' \textit{importance}. In contrast, we propose Uncertainty-guided Continual Bayesian Neural Networks (UCB), where the learning rate adapts according to the uncertainty defined in the probability distribution of the weights in networks. Uncertainty is a natural way to identify \textit{what to remember} and \textit{what to change} as we continually learn, and thus mitigate catastrophic forgetting. We also show a variant of our model, which uses uncertainty for weight pruning and retains task performance after pruning by saving binary masks per tasks. We evaluate our UCB approach extensively on diverse object classification datasets with short and long sequences of tasks and report superior or on-par performance compared to existing approaches. Additionally, we show that our model does not necessarily need task information at test time, i.e. it does not presume knowledge of which task a sample belongs to.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning (2021, methodology): “This uncertainty has been used to adapt the stepsizes during continual learning in Uncertainty-guided Continual BNNs (UCB) (Ebrahimi et al., 2020).”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Except for the TIL setting that always provides the task identity I D t [70], [71], [72], [73], other scenarios generally require the model to determine which θ ( t ) to use based on the input data, as shown in (10).”
Continual Learning via Sequential Function-Space Variational Inference (2023, background): “…regularization includes various modifications to VCL (Ahn et al., 2019; Kessler et al., 2019), uncertainty-guided continual learning in Bayesian neural networks (Ebrahimi et al., 2020), and a variation of SI known as asymmetric loss approximation with single-side overestimation (Park et al., 2019).”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “, penalize changes to model parameters) [2, 15, 30, 58, 71] or the prediction space (i.”
Continual Learning of Natural Language Processing Tasks: A Survey (2022, methodology): “Some others regularize the learning rate to slow down parameter updates or control the geometry of the local minima [Ebrahimi et al., 2020; Mirzadeh et al., 2020].”
Generative Negative Text Replay for Continual Vision-Language Pretraining (2022, methodology): “The first is the regularization methods [16,25], which penalize changes on significant weights of previously learned models.”
DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning (2022, methodology): “Regularization-based methods aim to penalize the update either in function space [8, 24] or in parameter space [2, 9, 18, 20, 22, 23, 32, 38, 45].”
General Incremental Learning with Domain-aware Categorical Representations (2022, methodology): “The first are the regularization-based methods [6, 10, 15], which add regularization on the parameters directly to prevent dramatic changes of the important parameters.”
Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference (2021, neutral): “(1) Regularization methods (Kirkpatrick et al., 2017; Zenke et al., 2017; Lee et al., 2017b; Ebrahimi et al., 2020) store previous task information in the form of model priors and use it for regularizing the neural network currently being trained.”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “Regularization based methods [23–26] generally impose restrictions on the update process of various model parameters and hyperparameters in order to consolidate previously learned knowledge while learning new tasks to mitigate catastrophic forgetting in continual learning.”
Kernel Continual Learning (2021, methodology): “Naturally, these methods can also be explored from the perspective of Bayesian optimization (Nguyen et al., 2018; Titsias et al., 2020; Schwarz et al., 2018; Ebrahimi et al., 2020; Ritter et al., 2018).”
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning (2021, background): “, penalize changes to model parameters) [1, 11, 28, 53, 61] or the prediction space (i.”
Gradient Projection Memory for Continual Learning (2021, methodology): “Datasets: We evaluate our continual learning algorithm on Permuted MNIST (PMNIST) (Le-cun et al., 1998), 10-Split CIFAR-100 (Krizhevsky, 2009), 20-Spilt miniImageNet (Vinyals et al., 2016) and sequence of 5-Datasets (Ebrahimi et al., 2020b).”
Continual Lifelong Learning in Natural Language Processing: A Survey (2020, methodology): “On a model level, a combination of CL with Bayesian principles should allow to identify better the importance of each parameter of an NN and aid parameter pruning and quantization (e.g. Ebrahimi et al., 2020; Golkar et al., 2019).”
Knowledge Capture and Replay for Continual Learning (2020, background): “, 2020] [Ebrahimi et al., 2020] are prevalent, algorithms based on rehearsal or replay strategies are gaining momentum.”
Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “They can either a Bayesian (Nguyen et al., 2017; Titsias et al., 2019; Schwarz et al., 2018; Ebrahimi et al., 2019; Ritter et al., 2018) or frequentist views (Farajtabar et al., 2019; He & Jaeger, 2018; Zeng et al., 2018).”
Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting (2020, methodology): “This simple method, known as experience replay, has been explored and shown to be effective (Rebufﬁ et al., 2017; Wu et al., 2019; Castro et al., 2018; Rajasegaran et al., 2020; Ebrahimi et al., 2020b; Hayes et al., 2019; Riemer et al., 2018).”
Meta-Consolidation for Continual Learning (2020, background): “This is also referred as single-head or multi-head evaluation in literature [13, 79, 21].”