Federated Learning over Wireless Networks: Optimization Model Design and Analysis
By Nguyen H. Tran, Wei Bao, Albert Y. Zomaya, Minh N. H. Nguyen, Choong Seon Hong. 2019
911 citations recorded by the source corpus.
There is an increasing interest in a new machine learning technique called Federated Learning, in which the model training is distributed over mobile user equipments (UEs), and each UE contributes to the learning model by independently computing the gradient based on its local training data. Federated Learning has several benefits of data privacy and potentially a large amount of UE participants with modern powerful processors and low-delay mobile-edge networks. While most of the existing work focused on designing learning algorithms with provable convergence time, other issues such as uncertainty of wireless channels and UEs with heterogeneous power constraints and local data size, are under-explored. These issues especially affect to various trade-offs: (i) between computation and communication latencies determined by learning accuracy level, and thus (ii) between the Federated Learning time and UE energy consumption. We fill this gap by formulating a Federated Learning over wireless network as an optimization problem FEDL that captures both trade-offs. Even though FEDL is non-convex, we exploit the problem structure to decompose and transform it to three convex sub-problems. We also obtain the globally optimal solution by charactering the closed-form solutions to all sub-problems, which give qualitative insights to problem design via the obtained optimal FEDL learning time, accuracy level, and UE energy cost. Our theoretical analysis is also illustrated by extensive numerical results.
Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Federated & Distributed Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dusist Niyato
- H. Vincent Poor
- Khaled Ben Letaief
- Mingzhe Chen
- Yuanming Shi
- Walid Saad
- Medhi Bennis
- Ming Ding
In Electrical and Electronic Engineering
- H. Vincent Poor
- Khaled Ben Letaief
- Rui Zhang
- Walid Saad
- Mingzhe Chen
- Dusist Niyato
- Medhi Bennis
- Yuanming Shi
In Federated & Distributed Learning
- Mingzhe Chen
- Walid Saad
- H. Vincent Poor
- Medhi Bennis
- Khaled Ben Letaief
- Yuanming Shi
- Dusist Niyato
- Shuguang Cui
Citations in the corpus
- Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy (2020, background): “• Due to data privacy legislation, banks [5] and telecom [6] companies cannot share individual records.”
- Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach (2019, background): “According to [4], the energy consumption by data owner n to transmit local model updates in a global iteration is expressed as E n = T com n · ρn = σρn B ln(1+ ρnhn N0 ) .”
- Cloud-based Federated Boosting for Mobile Crowdsensing (2020, background): “Instability of users is not handled by the federated learning architecture [14] or its follow-up apps on specific models [30] [22].”
- Client-Edge-Cloud Hierarchical Federated Learning (2019, methodology): “In the following, we provide the models for wireless communications and local computations [8].”
- Convergence Time Optimization for Federated Learning Over Wireless Networks (2020, background): “In [8] and [9], the authors investigated the tradeoff between the FL convergence time and users’ energy consumption.”
- Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, neutral): “In [266], the authors studied the trade-off between the energy cost of model updates and the latency in FEEL over fading channels.”
- Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach (2020, background): “Ek is regarded as a constant for all the vehicular clients [14], [24].”
- Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning (2020, methodology): “The authors of [18] adopt TDMA for the MAC layer, and jointly optimize the device CPU frequency, the transmission latency, and the local model accuracy to minimize the weighted sum of training latency and total device energy consumption.”
- Exploiting Unlabeled Data in Smart Cities using Federated Learning (2020, background): “Federated learning (FL) has emerged as an efficient technique to avoid privacy infringement while allowing the discovery of the necessary information hidden in the data [5].”
- Throughput-Optimal Topology Design for Cross-Silo Federated Learning (2020, background): “To the best of our knowledge, energetic considerations for federated learning have not been adequately explored, but for a few papers considering FL for mobile devices [42, 97].”
- Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks (2019, background): “To this end, there are some very recent works that focus on the distributed machine learning framework with a particular focus on wireless communications [1]–[3], [7]–[13].”
- A Learning-Based Incentive Mechanism for Federated Learning (2020, background): “[26] have studied the federated learning over wireless networks by formulating an optimization problem that capture tradeoff between communication and computation cost.”
- FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing (2020, background): “Federated learning has triggered various potential applications in the sectors of smart medicine, finance, and next-generation wireless communications [5, 6, 7].”
- Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, background): “The authors in [64], [65] study the trade-off between the local ML model updates and global ML model aggregation so as to minimize the total energy consumption for local ML model training and transmission or the FL training loss.”
- A Crowdsourcing Framework for On-Device Federated Learning (2019, background): “Setting a different motivation, FL recently has attracted an increasing interest [7], [11], [12], [13], [15], [22] in which collaboration of the number of devices with non-i.”