Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning

By Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, P. Heng. 2021

110 citations recorded by the source corpus.

The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones. To address such the 'sensitivity-stability' dilemma, most previous efforts have been contributed to minimizing the empirical risk with different parameter regularization terms and episodic memory, but rarely exploring the usages of the weight loss landscape. In this paper, we investigate the relationship between the weight loss landscape and sensitivity-stability in the continual learning scenario, based on which, we propose a novel method, Flattening Sharpness for Dynamic Gradient Projection Memory (FS-DGPM). In particular, we introduce a soft weight to represent the importance of each basis representing past tasks in GPM, which can be adaptively learned during the learning process, so that less important bases can be dynamically released to improve the sensitivity of new skill learning. We further introduce Flattening Sharpness (FS) to reduce the generalization gap by explicitly regulating the flatness of the weight loss landscape of all seen tasks. As demonstrated empirically, our proposed method consistently outperforms baselines with the superior ability to learn new skills while alleviating forgetting effectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Optimization & Training

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “When task i needs to accommodate another task j , the maximum sacriﬁce of its performance can be estimated by performing a second-order Taylor expansion of (cid:4) i ( θ ; D where where λ max i is the maximum eigenvalue of the Hessian matrix Note that the order of task i and j can be arbitrary, that”
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, methodology): “After disturbing the network parameters with random values, we evaluate the training error with cross-entropy [54], where each line is a different direction of disturbance.”
Incremental Task Learning with Incremental Rank Updates (2022, background): “It has been a popular continual learning setup even in the very recent literature [7,14,11,36,31,40].”
CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One (2022, positive): “Flatness of Loss Landscape provides a conceptual explanation of generalization for deep neural networks, which is recently introduced to understand catastrophic forgetting in continual learning [31,42,11,6,25].”
Forget-free Continual Learning with Winning Subnetworks (2022, methodology): “We report the mean and standard deviation of the average accuracy (ACC), average capacity (CAP), and average backward transfer (BWT) across 5 independent runs with 5 seeds under the same experimental setup (Deng et al., 2021).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “This can be achieved by exploiting the singular vector space of the activations of the network, which applies to linear and convolutional layers (Saha et al., 2021; Deng et al., 2021), as well as recurrent layers (Duncker et al., 2020).”