CMFL: Mitigating Communication Overhead for Federated Learning

By Luping Wang. IEEE International Conference on Distributed Computing Systems, 2019

454 citations recorded by the source corpus.

Federated Learning enables mobile users to collaboratively learn a global prediction model by aggregating their individual updates without sharing the privacy-sensitive data. As mobile devices usually have limited data plan and slow network connections to the central server where the global model is maintained, mitigating the communication overhead is of paramount importance. While existing works mainly focus on reducing the total bits transferred in each update via data compression, we study an orthogonal approach that identifies irrelevant updates made by clients and precludes them from being uploaded for reduced network footprint. Following this idea, we propose Communication-Mitigated Federated Learning (CMFL) in this paper. CMFL provides clients with feedback information regarding the global tendency of model updating. Each client checks if its update aligns with this global tendency and is relevant enough to model improvement. By avoiding uploading those irrelevant updates to the server, CMFL can substantially reduce the communication overhead while still guaranteeing the learning convergence. CMFL is shown to achieve general improvement in communication efficiency for almost all of the existing federated learning schemes. We evaluate CMFL through extensive simulations and EC2 emulations. Compared with vanilla Federated Learning, CMFL yields 13.97x communication efficiency in terms of the reduction of network footprint. When applied to Federated Multi-Task Learning, CMFL improves the communication efficiency by 5.7x with 4% higher prediction accuracy.

Subfields: Artificial Intelligence, Computer Networks and Communications, Transportation

View Paper: https://www.semanticscholar.org/paper/e928c7bbb0089abf591a2b4861d8885b6d762e5b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Transportation

Citations in the corpus

A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, background): “The bottleneck of FL’s communication overhead has been addressed in [49] by proposing a communicationmitigated FL (CMFL) solution.”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “Communication-Mitigated Federated Learning ( CMFL ) [143] avoids the transmission of irrelevant updates to the server by measuring whether local updates are consistent with global updates, which effectively decreases the overhead of communication transmission.”
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “[14] propose CMFL, a communicationmitigated federated learning approach that reduces communi-”
Federated Learning: Opportunities and Challenges (2021, background): “Existing works try to solve this by either data compression [17] or by allowing only the relevant outputs by the clients[13, 22] to be sent back to the central server.”
Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, background): “In addition, the bottleneck can be worsened due to (i) unreliable network conditions of participating devices [91] and (ii) the asymmetry in Internet connection speeds in which upload speed is faster than download speed, resulting in delays in model uploads by participants [89].”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, methodology): “based industrial IoT applications, many efficient FL solutions have been introduced to solve communication issues by using update reconstruction techniques [174], [175] or relevance detection during the model update [177].”
Federated Learning Meets Blockchain in Edge Computing: Opportunities and Challenges (2021, methodology): “A blockchain platform running on a Delegated Proof of Stake (DPoS) [38] consensus mechanism is adopted to establish the decentralized training network for FLchain in edge computing.”
HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning (2020, background): “The effect of different local iteration numbers L(θ) = [5, 10, 20, 25, 50] on convergence speed is exhibited in Fig.”