Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
By Takayuki Nishio, Ryo Yonetani. 2018
1,773 citations recorded by the source corpus.
We envision a mobile edge computing (MEC) framework for machine learning (ML) technologies, which leverages distributed client data and computation resources for training high-performance ML models while preserving client privacy. Toward this future goal, this work aims to extend Federated Learning (FL), a decentralized learning framework that enables privacy-preserving training of models, to work with heterogeneous clients in a practical cellular network. The FL protocol iteratively asks random clients to download a trainable model from a server, update it with own data, and upload the updated model to the server, while asking the server to aggregate multiple client updates to further improve the model. While clients in this protocol are free from disclosing own private data, the overall training process can become inefficient when some clients are with limited computational resources (i.e., requiring longer update time) or under poor wireless channel conditions (longer upload time). Our new FL protocol, which we refer to as FedCS, mitigates this problem and performs FL efficiently while actively managing clients based on their resource conditions. Specifically, FedCS solves a client selection problem with resource constraints, which allows the server to aggregate as many client updates as possible and to accelerate performance improvement in ML models. We conducted an experimental evaluation using publicly-available large-scale image datasets to train deep neural networks on MEC environment simulations. The experimental results show that FedCS is able to complete its training process in a significantly shorter time compared to the original FL protocol.
Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Federated & Distributed Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Medhi Bennis
- Peter Richtárik
- Jakub Konecný
- H. Vincent Poor
- Qiang Yang
- Jihong Park
- Dusist Niyato
- Virginia Smith
In Electrical and Electronic Engineering
- Medhi Bennis
- Merouane Debbah
- H. Vincent Poor
- Jihong Park
- Rui Zhang
- Walid Saad
- Dusist Niyato
- Khaled Ben Letaief
In Federated & Distributed Learning
- Jakub Konecný
- Peter Richtárik
- H. Vincent Poor
- Virginia Smith
- Medhi Bennis
- Qiang Yang
- Jianyu Wang
- Gauri Joshi
Citations in the corpus
- Gradient Descent with Compressed Iterates (2019, background): “This situation is common in large-scale distributed optimization involving many edge devices, and common challenges include data heterogeneity [37], privacy [7], resource management [25, 33], and system heterogeneity as well as communication efficiency [16, 18].”
- Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach (2019, background): “Considering clients with heterogeneous resources, the authors in [7] proposed a client selection scheme for federated learning based on a greedy algorithm.”
- Towards Federated Learning at Scale: System Design (2019, background): “Nishio & Yonetani (2018) focuses on applying FL in different environmental conditions, namely where the server can reach any subset of heterogeneous devices to initiate a round, but receives updates sequentially due to cellular bandwidth limit.”
- Wireless Network Intelligence at the Edge (2018, methodology): “FL has been shown to improve communication efficiency by MSI quantization [30], adjusting the MSI update period [28], and optimizing devices’ scheduling policy [31].”
- Federated Learning for Keyword Spotting (2018, background): “Several improvements to the initial FedAvg algorithm have been suggested with a focus on client selection [8], budget-constrained optimization [9] and upload cost reduction for clients [10].”
- Federated Learning: Challenges, Methods, and Future Directions (2019, background): “For example, Nishio and Yonetani [36] explore novel devicesampling policies based on systems resources, with the aim being for the server to aggregate as many device updates as possible within a predefined time window.”
- Adaptive Federated Learning in Resource Constrained Edge Computing Systems (2018, background): “Participant (client) selection for federated learning is studied in [22].”
- FedMD: Heterogenous Federated Learning via Model Distillation (2019, background): “There is system heterogeneity when each participant has a different amount of bandwidth and computational power; this was partly resolved by the native asynchronous scheme of federated learning, which was further refined e.g. to enable active sampling [6, 7] and improve fault tolerance [8].”
- Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks (2019, background): “Recognizing such criticality, a host of studies have been carried out and resulted in various scheduling protocols for FL, ranging from minimizing the transmission latency [13], maximizing the spectral utility [14], to opportunistically alternating between selecting the UEs with advantageous and dis”
- Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications (2018, background): “The recent work in [30], proposes a new FL protocol that solves a client selection problem with resource constraints in mobile edge computing.”
- A survey on federated learning: challenges and applications (2022, positive): “For responding to this issue, Nishio [81] et al. proposed a new FL protocol, Fedcs, where the deadline for downloading, updating and uploading local models was pre-set.”
- Towards Taming the Resource and Data Heterogeneity in Federated Learning (2019, background): “FedCS [13] proposes to solve data party selection issue via a deadline-based approach that filters out slowly-responding parties.”
- Federated Learning in Smart City Sensing: Challenges and Opportunities (2020, positive): “Nishio et al. [139] proposed FedCS, a Federated Learning protocol to improve client selection under heterogeneous device scenarios.”
- Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning (2019, neutral): “To reduce communication links, model updates can be sparsified by collecting model updates until a time deadline [16], based upon significant recent model changes [12], [17], or based on channel conditions [18]–[20].”
- Federated Learning in Edge Computing: A Systematic Survey (2022, methodology): “Second, a subset of current clients is chosen at random or using client selection algorithms such as Federated Client Selection (FedCS) [20].”
- Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training With Non-IID Private Data (2020, background): “An alternative strategy is to limit the number of participatingmobile users by selecting the users satisfying the stringent requirement for model update time[14], [15].”