Memory-Efficient Class-Incremental Learning for Image Classification

By Hanbin Zhao, Haibo Wang, Yongjian Fu, Fei Wu, Xi Li. 2020

108 citations recorded by the source corpus.

With the memory-resource-limited constraints, class-incremental learning (CIL) usually suffers from the “catastrophic forgetting” problem when updating the joint classification model on the arrival of newly added classes. To cope with the forgetting problem, many CIL methods transfer the knowledge of old classes by preserving some exemplar samples into the size-constrained memory buffer. To utilize the memory buffer more efficiently, we propose to keep more auxiliary low-fidelity exemplar samples, rather than the original real-high-fidelity exemplar samples. Such a memory-efficient exemplar preserving scheme makes the old-class knowledge transfer more effective. However, the low-fidelity exemplar samples are often distributed in a different domain away from that of the original exemplar samples, that is, a domain shift. To alleviate this problem, we propose a duplet learning scheme that seeks to construct domain-compatible feature extractors and classifiers, which greatly narrows down the above domain gap. As a result, these low-fidelity auxiliary exemplar samples have the ability to moderately replace the original exemplar samples with a lower memory cost. In addition, we present a robust classifier adaptation scheme, which further refines the biased classifier (learned with the samples containing distillation label knowledge about old classes) with the help of the samples of pure true class labels. Experimental results demonstrate the effectiveness of this work against the state-of-the-art approaches. We will release the code, baselines, and training statistics for all models to facilitate future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Qiwen Wang
5. Ziwei Liu
6. Zhiyuan Qi
7. Lanqing Hong
8. Zhi-Hong Qi

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Ziwei Liu
5. Ming-Hsuan Yang
6. Luc Van Gool
7. Zhenguo Li
8. Dacheng Tao

Citations in the corpus

• Generative Negative Text Replay for Continual Vision-Language Pretraining (2022, background): “Continual Learning Continual learning [54,13,43,48] aims to integrate novel knowledge in a sequential fashion where old data are usually unavailable.”
• When Video Classification Meets Incremental Classes (2021, background): “The same herding exemplars are also utilized in multiple works [5, 18, 19, 55, 62].”
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning (2024, background): “Data rehearsal-based methods [3, 6, 37, 45, 75] select and replay exemplars from former classes when learning new ones to recover former knowledge.”
• Class-Incremental Learning: A Survey (2023, background): “Reference § 3.1 Data Replay Direct Replay [26], [33], [34], [35], [36], [37], [38], [39], [40] Generative Replay [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52] § 3.2: Data Regularization [34], [53], [54], [55], [56], [57], [58] § 3.3 Dynamic Networks Neuron Expansion [45],…”
• Online Continual Learning through Mutual Information Maximization (2022, background): “…approach (Chaudhry et al., 2020; Castro et al., 2018; Re-buffi et al., 2017; de Masson d’Autume et al., 2019; Wu et al., 2019; Hou et al., 2019; Zhao et al., 2021; Korycki & Krawczyk, 2021; Sokar et al., 2021; Yan et al., 2021; Hu et al., 2021; Wang et al., 2022), which uses a memory buffer…”
• Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation (2023, methodology): “Our method distinguishes itself in four aspects: (1) We save a single compact prototype per class for better memory efficiency, unlike some prior works [14, 32] which store multiple prototypes or exemplars.”
• Deep Class-Incremental Learning: A Survey (2023, background): “1 Data Replay Direct Replay [35], [39], [40], [41], [42], [43], [44], [45], [46], [47]”
• Image De-raining via Continual Learning (2021, methodology): “The methods of overcoming catastrophic forgetting can be mainly divided into three categories: transfer learning approaches, rehearsal mechanisms, and parameter regularization methods [2, 15, 7, 28, 49, 21, 26, 1].”