FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
By Hong-You Chen, Wei-Lun Chao. International Conference on Learning Representations, 2020
342 citations recorded by the source corpus.
Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data. A crucial step is therefore to aggregate local models into a global model, which has been shown challenging when users have non-i.i.d. data. In this paper, we propose a novel aggregation algorithm named FedBE, which takes a Bayesian inference perspective by sampling higher-quality global models and combining them via Bayesian model Ensemble, leading to much robust aggregation. We show that an effective model distribution can be constructed by simply fitting a Gaussian or Dirichlet distribution to the local models. Our empirical studies validate FedBE's superior performance, especially when users' data are not i.i.d. and when the neural networks go deeper. Moreover, FedBE is compatible with recent efforts in regularizing users' model training, making it an easily applicable module: you only need to replace the aggregation method but leave other parts of your federated learning algorithm intact. Our code is publicly available at https://github.com/hongyouc/FedBE.
Subfields: Artificial Intelligence, Building and Construction, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/60ba0a0c600df527b87525469dcde42f1925cdcc
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Building and Construction
In Management Science and Operations Research
Citations in the corpus
- Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning (2022, methodology): “FedBE [3] generates a series of global models from Bayesian perspective using the local models, then summarizes these models into one global model by ensemble knowledge distillation.”
- A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “The focuses of FL research can be roughly divided into three aspects: (1) improving the efficiency and effectiveness of FL [24, 68, 87, 100, 120, 211, 213], (2) improving the security of FL to attacks that aim to undermine the integrity of FL models and degrade the model performance [9, 13, 185, 198”