Learn from Others and Be Yourself in Heterogeneous Federated Learning

By Wenke Huang, Mang Ye, Bo Du. Computer Vision and Pattern Recognition, 2022

325 citations recorded by the source corpus.

Federated learning has emerged as an important distributed learning paradigm, which normally involves collaborative updating with others and local updating on private data. However, heterogeneity problem and catastrophic forgetting bring distinctive challenges. First, due to non-i.i.d (identically and independently distributed) data and heterogeneous architectures, models suffer performance degradation on other domains and communication barrier with participants models. Second, in local updating, model is separately optimized on private data, which is prone to overfit current data distribution and forgets previously acquired knowledge, resulting in catastrophic forgetting. In this work, we propose FCCL (Federated CrossCorrelation and Continual Learning). For heterogeneity problem, FCCL leverages unlabeled public data for communication and construct cross-correlation matrix to learn a generalizable representation under domain shift. Mean- while, for catastrophic forgetting, FCCL utilizes knowledge distillation in local updating, providing inter and intra domain information without leaking privacy. Empirical results on various image classification tasks demonstrate the effectiveness of our method and the efficiency of modules.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/9787d29aa66966629350a5c0806e32a19882a2c9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bo Du
2. Mang Ye
3. Pong Chi Yuen
4. Dacheng Tao
5. Xiuwen Fang
6. Jing Zhang
7. Charu C. Aggarwal
8. Xinlei Chen

In Radiology, Nuclear Medicine and Imaging

1. Dacheng Tao
2. Bo Du
3. Pong Chi Yuen
4. Mang Ye

Citations in the corpus

• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “[48] RHFL uses irrelevant data for knowledge distillation, thereby solving the problem of model heterogeneity.”