Anti-Retroactive Interference for Lifelong Learning
By Runqi Wang, Yuxiang Bao, Baochang Zhang, Jianzhuang Liu, Wentao Zhu, Guodong Guo. 2022
30 citations recorded by the source corpus.
Humans can continuously learn new knowledge. However, machine learning models suffer from drastic dropping in performance on previous tasks after learning new tasks. Cognitive science points out that the competition of similar knowledge is an important cause of forgetting. In this paper, we design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. First, we disrupt the sample's background distribution through a background attack, which strengthens the model to extract the key features of each task. Second, according to the similarity between incremental knowledge and base knowledge, we design an adaptive fusion of incremental knowledge, which helps the model allocate capacity to the knowledge of different difficulties. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum. The proposed method is validated on the MNIST, CIFAR100, CUB200 and ImageNet100 datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Meta & Continual Learning
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “ARI [187] combines adversarial attacks with experience replay to obtain task-specific models, which are then fused together through meta-training.”
- Class-Incremental Learning: A Survey (2023, background): “[174] extends data replay with adversarial attack and meta-learns an adaptive fusion module to help allocate capacity to the knowledge of different difficulties.”