Federated Class-Incremental Learning

By Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, Qi Zhu. 2022

299 citations recorded by the source corpus.

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global class-incremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%~15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github.com/conditionWang/FCIL.

Subfields: Artificial Intelligence, Meta & Continual Learning, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Qiwen Wang
6. Jun Zhu
7. Ziwei Liu
8. Liyuan Wang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

In Federated & Distributed Learning

1. Weiming Zhuang
2. Chen Chen
3. Jie M. Zhang
4. LingJuan Lv

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “GLFC [95] overcomes catastrophic forgetting through knowledge distillation of a few old training samples, and alleviates class imbalance through gradient compensation.”
• Deep Class-Incremental Learning: A Survey (2023, methodology): “, semantic segmentation [164], [165], [166], [167], person re-identification [168], [169], human action recognition [95], and federated learning [137].”
• Class-Incremental Learning: A Survey (2023, methodology): “Moreover, since data regularization and data replay both need to save previous data in the memory, similar problems will also occur in data regularization-based methods, e.g. , overfitting, generalization issues [53], [162], and privacy concerns [149].”
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning (2024, background): “Class-Incremental Learning (CIL) : requires a learning system to continually absorb new class knowledge without forgetting existing ones [13, 14, 20, 22, 38, 57, 59, 74, 81, 82], which can be roughly divided into several categories.”
• TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation (2023, methodology): “[11] propose a exemplar-based method that stores historical data to address catastrophic forgetting.”