Addressing Class Imbalance in Federated Learning

By Lixu Wang, Shichao Xu, Xiao Wang, Qi Zhu. 2020

329 citations recorded by the source corpus.

Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients' side may lead to significant challenges such as class imbalance and non-IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or server, and previous methods for class imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting class imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition of training data for each FL round, and design a new loss function -- Ratio Loss to mitigate the impact of the imbalance. Our experiments demonstrate the importance of acknowledging class imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bingsheng He
2. Qinbin Li
3. Zhen Fang
4. Xiao Wang
5. Ying Shan
6. Dawn Song
7. Mohammad R. Hayajneh
8. Yujun Shi

Citations in the corpus

• Federated Class-Incremental Learning (2022, methodology): “Federated learning (FL) [4, 18, 42, 46] providing secure privacy protection for local clients.”
• Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models (2023, methodology): “To improve over FedAvg, previous works have either focused on local client training [18, 19, 20, 21, 22, 23, 24] or global server aggregation [25, 26, 27, 28, 29, 30].”
• Model-Contrastive Federated Learning (2021, background): “We also notice that there are other related contemporary work [1, 30, 43] when preparing this paper.”
• Federated Learning in Edge Computing: A Systematic Survey (2022, methodology): “Moreover, the authors of [180] designed new methods for detecting data imbalances in FL and mitigating their effects.”