Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation

By Canh T. Dinh, Nguyen H. Tran, Minh N. H. Nguyen, Choong Seon Hong, Wei Bao, Albert Y. Zomaya, Vincent Gramoli. IEEE/ACM Transactions on Networking, 2019

402 citations recorded by the source corpus.

There is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL), in which the model training is distributed over mobile user equipment (UEs), exploiting UEs’ local computation and training data. Despite its advantages such as preserving data privacy, FL still has challenges of heterogeneity across UEs’ data and physical resources. To address these challenges, we first propose FEDL, a FL algorithm which can handle heterogeneous UE data without further assumptions except strongly convex and smooth loss functions. We provide a convergence rate characterizing the trade-off between local computation rounds of each UE to update its local model and global communication rounds to update the FL global model. We then employ FEDL in wireless networks as a resource allocation optimization problem that captures the trade-off between FEDL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FEDL is non-convex, we exploit this problem’s structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights into problem design. Finally, we empirically evaluate the convergence of FEDL with PyTorch experiments, and provide extensive numerical results for the wireless resource allocation sub-problems. Experimental results show that FEDL outperforms the vanilla FedAvg algorithm in terms of convergence rate and test accuracy in various settings.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/6b256a0e98c5bb4687b84cc54738d185e7f4dbd4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Federated Learning for Intrusion Detection System: Concepts, Challenges and Future Directions (2021, background): “FL has the potential to utilize low-power IoT edge devices (i.e., low-power micro-controllers, Remote Telemetry Units, sensor nodes, etc.) for training [Dinh et al.(2019)Dinh, Tran, Nguyen, Hong, Bao, Zomaya, and Gramoli].”
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, methodology): “The work in [68] designed a FL algorithm”
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, neutral): “In [143], an enhanced FL algorithm was proposed to handle heterogeneous UE data and reduce the communication latency.”
A Crowdsourcing Framework for On-Device Federated Learning (2019, neutral): “Further, the authors in [43] have provided the convergence analysis to justify this relationship and the communication cost model, though with a different technique.”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, background): “Besides, the trade-off between the local computation rounds for local model updates and the global communication rounds for global model updates is characterized to guide the resource allocation for minimizing the total learning time and energy consumption [244].”
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, background): “Therefore, in order to strike a balance between energy cost and training time, the authors of [219] first design a new FL”
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning (2021, methodology): “We compare AutoFL with two closely related prior works: FedNova [121] and FEDL [26].”