Uncertainty-based Continual Learning with Adaptive Regularization

By Hongjoon Ahn, Donggyu Lee, Sungmin Cha, Taesup Moon. 2019

274 citations recorded by the source corpus.

We introduce a new neural network-based continual learning algorithm, dubbed as Uncertainty-regularized Continual Learning (UCL), which builds on traditional Bayesian online learning framework with variational inference. We focus on two significant drawbacks of the recently proposed regularization-based methods: a) considerable additional memory cost for determining the per-weight regularization strengths and b) the absence of gracefully forgetting scheme, which can prevent performance degradation in learning new tasks. In this paper, we show UCL can solve these two problems by introducing a fresh interpretation on the Kullback-Leibler (KL) divergence term of the variational lower bound for Gaussian mean-field approximation. Based on the interpretation, we propose the notion of node-wise uncertainty, which drastically reduces the number of additional parameters for implementing per-weight regularization. Moreover, we devise two additional regularization terms that enforce stability by freezing important parameters for past tasks and allow plasticity by controlling the actively learning parameters for a new task. Through extensive experiments, we show UCL convincingly outperforms most of recent state-of-the-art baselines not only on popular supervised learning benchmarks, but also on challenging lifelong reinforcement learning tasks. The source code of our algorithm is available at this https URL.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “…[149],[152],spacereservationforsubsequenttasks[115],[116], [256],etc. Ontheotherhand,solutiongeneralizabilitycould beexplicitlyimprovedbyoptimizingtheﬂatnessmetric[81],…”
Continual Learning via Sequential Function-Space Variational Inference (2023, background): “On the single-head version of split MNIST , a minimal coreset (one point per class, or two points per task) allows S - FSVI to outperform VCL and FROMP , both with coresets of 40 points per task (Table 1).”
Learnability and Algorithm for Continual Learning (2023, methodology): “Many existing approaches have used similar approaches (Ritter et al., 2018; Schwarz et al., 2018; Xu & Zhu, 2018; Castro et al., 2018; Dhar et al., 2019; Lee et al., 2019; Ahn et al., 2019; Liu et al., 2020a).”
A Theoretical Study on Solving Continual Learning (2022, methodology): “Using regularization [12, 13] to minimize changes to model parameters learned from previous tasks is a popular approach [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, background): “Many other papers use related approaches (Zenke, Poole, and Ganguli 2017; Fernando et al. 2017; Aljundi et al. 2017; Ritter, Botev, and Barber 2018; Xu and Zhu 2018; Kemker and Kanan 2018; Parisi et al. 2018; Ahn et al. 2019; Hu et al. 2021; Dhar et al. 2019; Adel, Zhao, and Turner 2020).”
Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks (2021, methodology): “UCL (Ahn et al., 2019) - a latest TCL method using a Bayesian online learning framework.”
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning (2021, neutral): “UCL [1] is a recent Task-CL method.”
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector (2021, neutral): “For example, EWC [21], SI [44], MAS [2] and UCL [1] explicitly added a regularization term to networks’ parameters.”
Representational Continuity for Unsupervised Continual Learning (2021, background): “The regularization approaches (Li & Hoiem, 2016; Zenke et al., 2017; Schwarz et al., 2018; Ahn et al., 2019) impose a regularization constraint to the objective that mitigates catastrophic forgetting.”
Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning (2021, background): “[1] introduces an interpretation of node-wise uncertainty on the Kullback-Leibler (KL) divergence term of the variational lower bound for Gaussian mean-field approximation.”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “As a result, learning the new task can minimally affect the performance of the model on the previous tasks.”
Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021, neutral): “Taking the third task as an example, we observe that the performance of conventional TDSs and UCL starts to decay sharply after learning new tasks, probably because the knowledge learned from these new tasks interferes with what was learned previously.”
Same State, Different Task: Continual Reinforcement Learning without Interference (2021, methodology): “RL environment, for example different games from the Arcade Learning Environment [8] or different simulated robotics environments [1].”
Continual Learning by Using Information of Each Class Holistically (2021, methodology): “There are also many TCL techniques (Fernando et al. 2017), e.g., GEM (Lopez-Paz and Ranzato 2017), A-GEM (Chaudhry et al. 2019), HAT (Serr`a et al. 2018), UCL (Ahn et al. 2019) and CAT (Ke, Liu, and Huang 2020).”
Generalized Variational Continual Learning (2020, methodology): “Ahn et al. (2019), like us, use regularizers based on the ELBO, but also measure importance on a per-node basis than a per-weight one.”
CPR: Classifier-Projection Regularization for Continual Learning (2020, methodology): “…be grouped into three main strategies: regularization methods (Li & Hoiem, 2017; Kirkpatrick et al., 2017; Zenke et al., 2017; Nguyen et al., 2018; Ahn et al., 2019; Aljundi et al., 2019), memory replay (Lopez-Paz & Ranzato, 2017; Shin et al., 2017; Rebuffi et al., 2017; Kemker & Kanan, 2018),…”
Variational Auto-Regressive Gaussian Processes for Continual Learning (2020, methodology): “hemes – regularization using a Laplace approximation and diagonal Fisher information matrix [15] or approximate path integral of gradient vector ﬁelds [42], using recursive variational approximations [21, 34, 1], targeting adaptive capacity in Bayesian neural networks [14], and episodic memory-based”
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization (2020, methodology): “To address this dilemma, a comprehensive study for neural network-based continual learning was conducted broadly under the following categories: regularization-based [18, 13, 41, 22, 1, 3], dynamic architecture-based [27, 39, 10], and replay memory-based [26, 20, 34, 11] methods.”