Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT

By Jed Mills, Jia Hu, Geyong Min. IEEE Internet of Things Journal, 2020

383 citations recorded by the source corpus.

The rapidly expanding number of Internet of Things (IoT) devices is generating huge quantities of data, but public concern over data privacy means users are apprehensive to send data to a central server for machine learning (ML) purposes. The easily changed behaviors of edge infrastructure that software-defined networking (SDN) provides makes it possible to collate IoT data at edge servers and gateways, where federated learning (FL) can be performed: building a central model without uploading data to the server. FedAvg is an FL algorithm which has been the subject of much study, however, it suffers from a large number of rounds to convergence with non-independent identically distributed (non-IID) client data sets and high communication costs per round. We propose adapting FedAvg to use a distributed form of Adam optimization, greatly reducing the number of rounds to convergence, along with the novel compression techniques, to produce communication-efficient FedAvg (CE-FedAvg). We perform extensive experiments with the MNIST/CIFAR-10 data sets, IID/non-IID client data, varying numbers of clients, client participation rates, and compression rates. These show that CE-FedAvg can converge to a target accuracy in up to <inline-formula> <tex-math notation="LaTeX">$\mathbf {6\times }$ </tex-math></inline-formula> less rounds than similarly compressed FedAvg, while uploading up to <inline-formula> <tex-math notation="LaTeX">$\mathbf {3\times }$ </tex-math></inline-formula> less data, and is more robust to aggressive compression. Experiments on an edge-computing-like testbed using Raspberry Pi clients also show that CE-FedAvg is able to reach a target accuracy in up to <inline-formula> <tex-math notation="LaTeX">$\mathbf {1.7\times }$ </tex-math></inline-formula> less real time than FedAvg.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/4f9477b4e894e05b6570e4ad7da82420085eb8c0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning (2022, background): “Recently, Federated Learning (FL) [30,31] has been proposed to replace the traditional heavily centralized learning paradigm and protect data privacy.”
Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research (2021, positive): “For example, a communication-efﬁcient approach based FL is proposed in [141] to efﬁciently process IoT data at the edge, but with data privacy preservation.”
Deep-Learning-Enhanced Human Activity Recognition for Internet of Healthcare Things (2020, methodology): “, accelerometer and gyroscope) using wireless sensor networks [2].”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “Client dropout refers to the problem of clients withdrawing from the federated training [135].”
A Learning-Based Incentive Mechanism for Federated Learning (2020, background): “This concept of federated learning is initialized by Google [4] and later has attracted great research attentions [25]–[28].”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, methodology): “[174] Communicationefficient FL HFL NN Edge devices Cloud server A communication-efficient FL approach called CE-FedAvg for optimizing communication rounds.”
Federated Learning on Non-IID Data: A Survey (2021, methodology): “A large body of research to address the above challenges, including reducing communication cost [18, 95, 97, 142], FL considering hardware constraints [28, 85], and additional protections against adversarial attacks [61, 160, 170].”
Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges (2021, background): “However, the comparison of delay performances of the proposed FLchain and advanced FL schemes in edge computing [37] has not been implemented.”
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, methodology): “[63] improve the federated averaging algorithm by using Adam optimization and a com-”