FedScale: Benchmarking Model and System Performance of Federated Learning at Scale

By Fan Lai, Yinwei Dai, Sanjay Sri Vallabh Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha V. Madhyastha, Mosharaf Chowdhury. International Conference on Machine Learning, 2021

302 citations recorded by the source corpus.

We present FedScale, a federated learning (FL) benchmarking suite with realistic datasets and a scalable runtime to enable reproducible FL research. FedScale datasets encompass a wide range of critical FL tasks, ranging from image classification and object detection to language modeling and speech recognition. Each dataset comes with a unified evaluation protocol using real-world data splits and evaluation metrics. To reproduce realistic FL behavior, FedScale contains a scalable and extensible runtime. It provides high-level APIs to implement FL algorithms, deploy them at scale across diverse hardware and software backends, and evaluate them at scale, all with minimal developer efforts. We combine the two to perform systematic benchmarking experiments and highlight potential opportunities for heterogeneity-aware co-optimizations in FL. FedScale is open-source and actively maintained by contributors from different institutions at http://fedscale.ai. We welcome feedback and contributions from the community.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/f356b19619e2d9ebeb96839f0e87a07071248179

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lichao Sun
2. Philip S. Yu
3. Charu C. Aggarwal
4. Bing Liu
5. Benjamin C. M. Fung
6. Ke Wang
7. Yongfeng Zhang
8. B. Liu

In Computer Vision and Pattern Recognition

1. Philip S. Yu
2. Lichao Sun
3. Charu C. Aggarwal
4. Pavel Berkhin

In Electrical and Electronic Engineering

1. Philip S. Yu
2. Lichao Sun

Citations in the corpus

• Decentralized Federated Learning: A Survey and Perspective (2023, background): “…values indicating the relative strength of the respective FL approach in that domain. metrics, which are the focal points of current state-of-the-art (SOTA) research and worthy of further investigation [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48].”