One-Shot Federated Learning

By Neel Guha, Ameet Talwalkar, Virginia Smith. 2019

268 citations recorded by the source corpus.

We present one-shot federated learning, where a central server learns a global model over a network of federated devices in a single round of communication. Our approach - drawing on ensemble learning and knowledge aggregation - achieves an average relative gain of 51.5% in AUC over local baselines and comes within 90.1% of the (unattainable) global ideal. We discuss these methods and identify several promising directions of future work.

Subfields: Artificial Intelligence, Computer Science Applications, Meta & Continual Learning, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Meta & Continual Learning

In Federated & Distributed Learning

Citations in the corpus

Personalized Federated Learning with Moreau Envelopes (2020, background): “Subsequently, one-shot FL [9] allows the global model to learn in one single round of communication.”
Personalized Federated Learning: A Meta-Learning Approach (2020, background): “Later, Guha et al. (2019) proposed one-shot Federated Learning (FL) in which the master node learns the model after a single round of communication.”
Salvaging Federated Learning by Local Adaptation (2020, background): “Alternative aggregation schemes (Yurochkin et al., 2019; Guha et al., 2019; Hsu et al., 2019) for various ﬂavors of federated learning provide neither privacy, nor robustness.”
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization (2019, background): “Guha et al. [2019] proposed one-shot federated learning in which the master node learns the model after a single round of communication.”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “Similarly, one-shot/few-shot heuristics [44, 45, 134] have recently been proposed for the federated setting, but have yet to be theoretically analyzed or evaluated at scale.”
Distilled One-Shot Federated Learning (2020, background): “While effective, breaking from this pattern offers many benefits, such as low communication [9] or private model architectures [15].”
On the Convergence of Local Descent Methods in Federated Learning (2019, background): “[4, 7] study the empirical performance of federated optimization.”
Federated Learning for Open Banking (2021, background): “The authors in [15] leverage distillation in a semi-supervised setting to reduce the size of the global model.”
Federated Learning: Opportunities and Challenges (2021, background): “Recently, there is a growing interest in one-shot federated learning which is first introduced by [12], where the global model is learned in a single round of communication.”
FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing (2020, unknown): “Some recent works include [16], [17].”
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications (2020, methodology): “11(a), by applying XorMixup to a one-shot FL framework having only one communication round [188], [189], each device in XorMixFL uploads its encoded seed samples to a server.”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “The authors of [95] leverage a one-shot federated learning approach which enables the server to learn a global model over a set of clients in one single communication iteration.”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, background): “The massive number of communication rounds between the central server and the clients are substituted with only one round in [56] to overcome the critical bottleneck”
Federated Learning for Healthcare Informatics (2019, methodology): “Guha et al. [35] focused on techniques for one-shot federated learning, in which they learn a global model from data in the network using only a single round of communication between the devices and”
Federated Learning on Non-IID Data Silos: An Experimental Study (2021, background): “There are some studies [18, 35] that propose FL algorithms using a single communication round.”
Think Locally, Act Globally: Federated Learning with Local and Global Representations (2020, methodology): “Recent methods aim to improve the efﬁciency of federated learning [13], perform learning in a one-shot setting [31], propose realistic benchmarks [14], and reduce the data mismatch between local and global data distributions [55].”
A Survey on Federated Learning for Resource-Constrained IoT Devices (2021, unknown): “The protocol outcome enables multiple data owners to collaboratively agree to generate a function without leaking any information [130]–[132].”
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning (2020, methodology): “We identify one major reason: the ensemble prediction ˆ p j can be noisy (e.g., arg max c ˆ p j [ c ] is not the true label of x j ), especially in the early rounds of FL.”