Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems
By Moming Duan, Duo Liu, Xianzhang Chen, Renping Liu, Yujuan Tan, Liang Liang. IEEE Transactions on Parallel and Distributed Systems, 2021
394 citations recorded by the source corpus.
Federated learning (FL) is a distributed deep learning method that enables multiple participants, such as mobile and IoT devices, to contribute a neural network while their private training data remains in local devices. This distributed approach is promising in the mobile systems where have a large corpus of decentralized data and require high privacy. However, unlike the common datasets, the data distribution of the mobile systems is imbalanced which will increase the bias of model. In this article, we demonstrate that the imbalanced distributed training data will cause an accuracy degradation of FL applications. To counter this problem, we build a self-balancing FL framework named Astraea, which alleviates the imbalances by 1) Z-score-based data augmentation, and 2) Mediator-based multi-client rescheduling. The proposed framework relieves global imbalance by adaptive data augmentation and downsampling, and for averaging the local imbalance, it creates the mediator to reschedule the training of clients based on Kullback–Leibler divergence (KLD) of their data distribution. Compared with FedAvg, the vanilla FL algorithm, Astraea shows +4.39 and +6.51 percent improvement of top-1 accuracy on the imbalanced EMNIST and imbalanced CINIC-10 datasets, respectively. Meanwhile, the communication traffic of Astraea is reduced by 75 percent compared to FedAvg.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/bdef93491b2eec2e71b9ba14fd304ff5eebf5188
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Han Yu
- Mohamed Adel Serhani
- Mohammad R. Hayajneh
- Haftay Gebreslasie Abreha
- Qiang Yang
- Li-zhen Cui
- Alysa Ziying Tan
- Mingxia Liu
Citations in the corpus
- Towards Personalized Federated Learning (2021, methodology): “System scalability is evaluated in terms of performance on a large number of clients [32], total elapse time [23, 29] and total memory consumption [29, 65].”
- Federated Learning in Edge Computing: A Systematic Survey (2022, background): “The authors of [179] highlighted the effects of the imbalanced distribution and developed Astraea’s self-balancing FL framework, which mitigates the imbalances.”