Class-Incremental Learning: A Survey

By Da-Wei Zhou, Qiwen Wang, Zhiyuan Qi, Han-Jia Ye, De-chuan Zhan, Ziwei Liu. 2023

380 citations recorded by the source corpus.

Deep models, e.g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world. However, novel classes emerge from time to time in our ever-changing world, requiring a learning system to acquire new knowledge continually. Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally and build a universal classifier among all seen classes. Correspondingly, when directly training the model with new class instances, a fatal problem occurs — the model tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. There have been numerous efforts to tackle catastrophic forgetting in the machine learning community. In this paper, we survey comprehensively recent advances in class-incremental learning and summarize these methods from several aspects. We also provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms empirically. Furthermore, we notice that the current comparison protocol ignores the influence of memory budget in model storage, which may result in unfair comparison and biased results. Hence, we advocate fair comparison by aligning the memory budget in evaluation, as well as several memory-agnostic performance measures.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Label-Efficient Incremental Learning: A Survey (2023, background): “While there have been few studies that surveys conventional incremental learners [Masana et al., 2020; De Lange et al., 2021; Wang et al., 2023; Zhou et al., 2023], their focus is still on the old label-hungry paradigm in incremental learning.”
Elastic Feature Consolidation for Cold Start Exemplar-free Incremental Learning (2024, neutral): “16) as discussed by Zhou et al. (2023); Castro et al. (2018).”
A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning (2022, background): “, 2021), and we refer the readers to (Zhou et al., 2023) for a holistic review.”
Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, background): “According to the target space, there usually exist logits-based, feature-based, and relation-based KD meth-ods [74].”
A Survey on Few-Shot Class-Incremental Learning (2023, background): “Zhou et al. (2023) summarized the latest progress in deep Class-IL from three aspects: data, model, and algorithm.”
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models (2025, unknown): “CL methods (De Lange et al., 2021; Masana et al., 2022; Wang et al., 2024; Zhou et al., 2024) focus primarily on reducing catastrophic forgetting of old classes after learning new classes.”