Optimizing Federated Learning on Non-IID Data with Reinforcement Learning

By Hao Wang, Zakhary Kaplan, Di Niu, Baochun Li. IEEE Conference on Computer Communications, 2020

1,034 citations recorded by the source corpus.

The widespread deployment of machine learning applications in ubiquitous environments has sparked interests in exploiting the vast amount of data stored on mobile devices. To preserve data privacy, Federated Learning has been proposed to learn a shared model by performing distributed training locally on participating devices and aggregating the local models into a global one. However, due to the limited network connectivity of mobile devices, it is not practical for federated learning to perform model updates and aggregation on all participating devices in parallel. Besides, data samples across all devices are usually not independent and identically distributed (IID), posing additional challenges to the convergence and speed of federated learning. In this paper, we propose Favor, an experience-driven control framework that intelligently chooses the client devices to participate in each round of federated learning to counterbalance the bias introduced by non-IID data and to speed up convergence. Through both empirical and mathematical analysis, we observe an implicit connection between the distribution of training data on a device and the model weights trained based on those data, which enables us to profile the data distribution on that device based on its uploaded model weights. We then propose a mechanism based on deep Q-learning that learns to select a subset of devices in each communication round to maximize a reward that encourages the increase of validation accuracy and penalizes the use of more communication rounds. With extensive experiments performed in PyTorch, we show that the number of communication rounds required in federated learning can be reduced by up to 49% on the MNIST dataset, 23% on FashionMNIST, and 42% on CIFAR-10, as compared to the Federated Averaging algorithm.

Subfields: Artificial Intelligence, Computer Science Applications, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/82f7073b503143cf3ed4d59dca2f206ba18a31b9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hangyu Zhu
2. Bo Du
3. Mang Ye
4. Pong Chi Yuen
5. Shiqing Liu
6. Jinjin Xu
7. Yaochu Jin
8. Dacheng Tao

In Computer Science Applications

1. Shiqing Liu
2. Jinjin Xu
3. Yaochu Jin
4. Hangyu Zhu
5. Dacheng Tao
6. Azzam Mourad
7. Bo Du
8. Tarik Taleb

In Sociology and Political Science

1. Dacheng Tao
2. Azzam Mourad
3. Haibing Guan
4. Bo Du
5. Tarik Taleb
6. Hadi Otrok
7. Omar Abdel Wahab
8. Mang Ye

Citations in the corpus

• Federated Learning on Non-IID Data: A Survey (2021, background): “Parametric models Non-IID data does affect the learning performance on parametric models [95, 166, 112, 129, 119, 7] and will always cause global model divergence in horizontal FL.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “Favor [209] is an experience-driven control framework that actively selects the best subset of clients to participate in federated learning iterations.”
• Holistic Network Virtualization and Pervasive Network Intelligence for 6G (2023, methodology): “A few end user selection algorithms are proposed based on principles such as training data distribution [195] and local training latency [196]; (2) FL parameters - To alleviate communication overhead, end users conduct a few local model updates before model uploading.”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “In [73], the authors propose an experience-based federated learning framework called FAVOR whose main purpose is to counterbalance the effect of”
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning (2022, methodology): “1 for the default heterogeneous setting (Lin et al. 2020; Wang et al. 2020c).”