Long-Tailed Class Incremental Learning

By Xialei Liu, Xuezhi Cao, Andrew D. Bagdanov, Ke Li, Mingg-Ming Cheng. 2022

60 citations recorded by the source corpus.

In class incremental learning (CIL) a model must learn new classes in a sequential manner without forgetting old ones. However, conventional CIL methods consider a balanced distribution for each new task, which ignores the prevalence of long-tailed distributions in the real world. In this work we propose two long-tailed CIL scenarios, which we term ordered and shuffled LT-CIL. Ordered LT-CIL considers the scenario where we learn from head classes collected with more samples than tail classes which have few. Shuffled LT-CIL, on the other hand, assumes a completely random long-tailed distribution for each task. We systematically evaluate existing methods in both LT-CIL scenarios and demonstrate very different behaviors compared to conventional CIL scenarios. Additionally, we propose a two-stage learning baseline with a learnable weight scaling layer for reducing the bias caused by long-tailed distribution in LT-CIL and which in turn also improves the performance of conventional CIL due to the limited exemplars. Our results demonstrate the superior performance (up to 6.44 points in average incremental accuracy) of our approach on CIFAR-100 and ImageNet-Subset. The code is available at https://github.com/xialeiliu/Long-Tailed-CIL

Subfields: Evaluation & Benchmarks, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

1. Liyuan Wang
2. Jun Zhu
3. De-chuan Zhan
4. Hang Su
5. Da-Wei Zhou
6. Ziwei Liu
7. Qiwen Wang
8. Han-Jia Ye

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Besides, other practical challenges have been considered and incorporated, such as multiple labels [215], noisy labels [29], [216], hierarchical granularity [2] and sub-populations [254], mixture of task similarity [207], long-tailed distribution [267], domain alignment [256],”
• Deep Class-Incremental Learning: A Survey (2023, background): “[212], [213], [214] propose to tackle the long-tailed class-incremental learning problem, where the head classes are easy to collect with adequate instances while tail classes are scarce.”
• Online Continual Learning For Interactive Instruction Following Agents (2024, methodology): “To explore a data-imbalance scenario (Liu et al., 2022; He et al., 2023), we remove the subsampling process in Sec.”
• Class-Incremental Learning: A Survey (2023, background): “[28], [141] propose to tackle the long-tailed CIL problem, where the head classes are easy to collect with adequate instances while tail classes are scarce.”