SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead

By Wentai Wu, Ligang He, Weiwei Lin, Rui Mao, Carsten R. Maple, S. Jarvis. 2019

427 citations recorded by the source corpus.

Federated learning (FL) has attracted increasing attention as a promising approach to driving a vast number of end devices with artificial intelligence. However, it is very challenging to guarantee the efficiency of FL considering the unreliable nature of end devices while the cost of device-server communication cannot be neglected. In this article, we propose SAFA, a semi-asynchronous FL protocol, to address the problems in federated learning such as low round efficiency and poor convergence rate in extreme conditions (e.g., clients dropping offline frequently). We introduce novel designs in the steps of model distribution, client selection and global aggregation to mitigate the impacts of stragglers, crashes and model staleness in order to boost efficiency and improve the quality of the global model. We have conducted extensive experiments with typical machine learning tasks. The results demonstrate that the proposed protocol is effective in terms of shortening federated round duration, reducing local resource wastage, and improving the accuracy of the global model at an acceptable communication cost.

Subfields: Artificial Intelligence, Computer Networks and Communications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Distilled One-Shot Federated Learning (2020, background): “SAFA takes a semi-synchronous approach; only up-to-date and deprecated clients synchronize with the server [34].”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “For example, the authors of [103] propose a Semi-Asynchronous Federated Averaging (SAFA) protocol that extends the FedAvg model.”
Server Averaging for Federated Learning (2021, background): “Safa et. al. explore iterate averaging in the context of block-cyclic SGD [15].”
A Survey on federated learning* (2020, background): “[33].”
Federated Learning with Buffered Asynchronous Aggregation (2021, background): “, 2020), requiring information about participating clients (Wu et al., 2020), frequently broadcasting the server model updates to all participating clients (Chen et al.”
A review of applications in federated learning (2020, background): “Considering trade-off between advantages of synchronous update and asynchronous training, Wu et al., (2019) proposed a semiasynchronous protocol which allow straggling clients don’t always go together with central server.”
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, background): “These bottlenecks in FL can potentially be addressed by asynchronous training mechanisms [280]–[282].”