A Field Guide to Federated Optimization

By Jianyu Wang, Zachary B. Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Agüera y Arcas, Maruan Al-Shedivat, Galen Andrew, S. Avestimehr, Katharine Daly and 43 more. 2021

487 citations recorded by the source corpus.

Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection. The distributed learning process can be formulated as solving federated optimization problems, which emphasize communication efficiency, data heterogeneity, compatibility with privacy and system requirements, and other constraints that are not primary considerations in other problem settings. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms through concrete examples and practical implementation, with a focus on conducting effective simulations to infer real-world performance. The goal of this work is not to survey the current literature, but to inspire researchers and practitioners to design federated learning algorithms that can be used in various practical applications.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

Citations in the corpus

Robust Aggregation for Federated Learning (2019, background): “While the exact benefits (or lack thereof) of local steps is an active area of research, local steps have been found empirically to reduce the amount of communication required for a moderately accurate solution [66, 84].”
Towards Out-Of-Distribution Generalization: A Survey (2021, background): “We direct readers to [115, 256] for a more comprehensive survey.”
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning (2022, background): “The readers may refer to monographs [17, 37] and the reference therein to follow up its recent advances.”
FairFed: Enabling Group Fairness in Federated Learning (2021, unknown): “Federated learning allows for decentralized training of large-scale models without requiring direct access to clients’ data, hence maintaining their privacy (Kairouz et al. 2021; Wang et al. 2021a).”
Federated Learning with Buffered Asynchronous Aggregation (2021, methodology): “Asynchronous FL. Asynchronous FL methods are a good match for cross-device FL settings, where clients have diﬀerent compute power and intermittent availability (Wang et al., 2021).”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “Section II: CommunicationEfficient Edge Training Section II-A: Edge Learning Models and Algorithms Federated Learning Horizontal and vertical federated learning [67], [34], [33]; federated optimization [68]”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, background): “We also note that user-level protection often is explored in context of Federated Learning (Kairouz et al., 2021d; Wang et al., 2021).”