AFEC: Active Forgetting of Negative Transfer in Continual Learning

By Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, Yi Zhong. 2021

123 citations recorded by the source corpus.

Continual learning aims to learn a sequence of tasks from dynamic data distributions. Without accessing to the old training samples, knowledge transfer from the old tasks to each new task is difficult to determine, which might be either positive or negative. If the old knowledge interferes with the learning of a new task, i.e., the forward knowledge transfer is negative, then precisely remembering the old tasks will further aggravate the interference, thus decreasing the performance of continual learning. By contrast, biological neural networks can actively forget the old knowledge that conflicts with the learning of a new experience, through regulating the learning-triggered synaptic expansion and synaptic convergence. Inspired by the biological active forgetting, we propose to actively forget the old knowledge that limits the learning of new tasks to benefit continual learning. Under the framework of Bayesian continual learning, we develop a novel approach named Active Forgetting with synaptic Expansion-Convergence (AFEC). Our method dynamically expands parameters to learn each new task and then selectively combines them, which is formally consistent with the underlying mechanism of biological active forgetting. We extensively evaluate AFEC on a variety of continual learning benchmarks, including CIFAR-10 regression tasks, visual classification tasks and Atari reinforcement tasks, where AFEC effectively improves the learning of new tasks and achieves the state-of-the-art performance in a plug-and-play way.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Of note, the above analyses are mainly based on finding a shared solution for all incremental tasks, which is subject to severe inter-task interference [353], [429], [430].”
Incorporating neuro-inspired adaptability for continual learning in artificial intelligence (2023, background): “Our preliminary versions of some individual components have been presented at top conferences in artificial intelligence [45, 47], while the current approach enjoys substantial extensions in terms of technical robustness, synergistic cooperation and biological plausibility (Supplementary D).”
The Ideal Continual Learner: An Agent That Never Forgets (2023, methodology): “Furthermore, their approaches are dual to each other and are the two sides of the same coin: Wang et al. (2021b) (approximately) maintains a basis for ∩t−1i=1null(X (j) i ) and Saha et al. (2021) for the orthogonal complement of ∩t−1i=1null(X (j) i ); see Appendix G for another example of this…”
SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model (2023, background): “Representative strategies include regularization-based approaches [19, 1, 50, 25, 40, 7], which preserve the old model and selectively stabilize changes of parameters or predictions; replay-based approaches [48, 29, 3, 43, 39, 38], which approximate and recover the previously-learned data…”
Genetic dissection of mutual interference between two consecutive learning tasks in Drosophila (2022, background): “Continual learning is a natural ability of animals ranging from invertebrates to vertebrates but a great challenge for artificial intelligence (Fayek, Cavedon, & Wu, 2020; Kudithipudi et al., 2022; Parisi, Kemker, Part, Kanan, & Wermter, 2019; Wang et al., 2021).”
CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One (2022, positive): “AFEC [45] encouraged the network parameters to resemble the optimal solution for each new task to mitigate potential negative transfer, while CPR [6] added a regularization term that maximized the entropy of output probability to find a flat minima.”
FOSTER: Feature Boosting and Compression for Class-Incremental Learning (2022, background): “The real world is constantly changing, with new concepts and categories continuously springing up [14,35,46,48].”
Memory Replay with Data Compression for Continual Learning (2022, positive): “…performance in class-incremental learning (Liu et al., 2021a; Hu et al., 2021) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al., 2020), few-shot (Tao et al., 2020b), semi-supervised (Wang et al., 2021a), and unsupervised continual learning (Khare et al., 2021).”
Continual Learning, Fast and Slow (2021, background): “In many real-world applications, data in continual learning may contain vastly different visual features, which could even be adversarial to one another [72].”
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, unknown): “Regularization-based methods [26, 32, 39, 61] mitigate catastrophic forgetting by introducing explicit regularization terms to balance new and old tasks.”
CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation (2023, methodology): “They can be roughly categorized into three groups: 1) Regularization-based methods [28, 2, 74, 64, 7, 37] limit the plasticity of the model to address catastrophic forgetting by regularizing important parameters or knowledge distillation.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, methodology): “A recent approach uses EWC with the Fisher information of the current task only, to forget knowledge of past tasks that prevents further learning (Wang et al., 2021); this method was used in supervised and RL tasks.”
A Unified and General Framework for Continual Learning (2024, unknown): “(Kirkpatrick et al., 2017; Zenke et al., 2017b; Chaudhry et al., 2018; Aljundi et al., 2018; Cha et al., 2021; Wang et al., 2021; Yang et al., 2023a).”