A Performance Evaluation of Federated Learning Algorithms

By Adrian Nilsson, Simon Smith, Gregor Ulm, Emil Gustavsson, Mats Jirstrand. DIDL@Middleware, 2018

325 citations recorded by the source corpus.

Federated learning is an approach to distributed machine learning where a global model is learned by aggregating models that have been trained locally on data-generating clients. Contrary to centralized optimization, clients can be very large in number and face challenges of data and network heterogeneity. Examples of clients include smartphones and connected vehicles, which highlights the practical relevance of federated learning. We benchmark three federated learning algorithms and compare their performance against a centralized approach where data resides on the server. The algorithms Federated Averaging (FedAvg), Federated Stochastic Variance Reduced Gradient, and CO-OP are evaluated on the MNIST dataset, using both i.i.d. and non-i.i.d. partitionings of the data. Our results show that FedAvg achieves the highest accuracy among the federated algorithms, regardless of how data was partitioned. Our comparison between FedAvg and centralized learning shows that they are practically equivalent when i.i.d. data is used. However, the centralized approach outperforms FedAvg with non-i.i.d. data.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/3d95b2fdd4babe475134324f49c5453580fd2993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Variational Federated Multi-Task Learning (2019, background): “Federated Averaging ( FedAvg ) [30, 33] is the state-of-the-art for federated learning with non-convex models and requires all the clients to share the same model.”
Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism (2019, methodology): “To enable FL, numerous optimization schemes, such as federated averaging (FedAvg) and FedProx can be used to train non-convex FL models [7].”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, background): “Indeed, the well-known FL algorithm FedAvg makes limited assumption that all IoT device clients join in each communication round for FL training [192], which is often not feasible in realistic IoT scenarios due to device connection loss or running out of battery.”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, methodology): “[129] propose a method utilizing correlated t-test to compare between different types of federated learning algorithms while bypass-”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, methodology): “[52] presented the aforementioned optimization algorithms: FedAvg—federated averaging [3],”
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System (2019, background): “A prominent direction of research in this domain is based on the weighted averaging of the model parameters [3, 18].”
Think Locally, Act Globally: Federated Learning with Local and Global Representations (2020, background): “This relies on the global model being sufficiently compact so that the parameters and updates can be sent efficiently over existing communication channels such as wireless networks (Nilsson et al., 2018).”