Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data
By Yujin Tang, Yifan Peng, Weishi Zheng. 2022
37 citations recorded by the source corpus.
Deep neural network (DNN) suffers from catastrophic forgetting when learning incrementally, which greatly limits its applications. Although maintaining a handful of samples (called “exemplars”) of each task could alleviate forgetting to some extent, existing methods are still limited by the small number of exemplars since these exemplars are too few to carry enough task-specific knowledge, and therefore the forgetting remains. To overcome this problem, we propose to “imagine” diverse counterparts of given exemplars referring to the abundant semantic-irrelevant information from unlabeled data. Specifically, we develop a learnable feature generator to diversify exemplars by adaptively generating diverse counterparts of exemplars based on semantic information from exemplars and semantically-irrelevant information from unlabeled data. We introduce semantic contrastive learning to enforce the generated samples to be semantic consistent with exemplars and perform semantic-decoupling contrastive learning to encourage diversity of generated samples. The diverse generated samples could effectively prevent DNN from forgetting when learning new tasks. Our method does not bring any extra inference cost and outperforms state-of-the-art methods on two benchmarks CIFAR-100 and ImageNet-Subset by a clear margin.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hang Su
- Jun Zhu
- Liyuan Wang
- Xingxing Zhang
- Tianyu Pang
- Yinpeng Dong
- Vamsi Krishna Ithapu
- Mirella Lapata
In Representation & Self-Supervised Learning
In Meta & Continual Learning
- Xingxing Zhang
- Liyuan Wang
- Hang Su
- Jun Zhu
- Simone Scardapane
- Valerio Marsocci
- Liyuan Wang
- Hongwei Yan
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Besides, other work considers a slightly different setting, where an additional unlabeled dataset is used for knowledge distillation (GD [239]) and data augmentation (L2I [402]), in order to facilitate supervised continual learning.”