Advances and Open Problems in Federated Learning
By Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Medhi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary B. Charles, Graham Cormode, Rachel Cummings and 48 more. Found. Trends Mach. Learn., 2019
9,019 citations recorded by the source corpus.
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches. Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges.
Subfields: Artificial Intelligence, Computer Science Applications, Federated & Distributed Learning
View Paper: https://arxiv.org/abs/1912.04977
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Qiang Yang
- Dacheng Tao
- Yoshua Bengio
- Peter Richtárik
- Martin Jaggi
- Mohit Bansal
- Quoc V. Le
In Computer Science Applications
- Zachary B. Charles
- Ling Liu
- Om Thakkar
- Gauri Joshi
- Jianyu Wang
- H. Brendan McMahan
- Bingsheng He
- Qinbin Li
In Federated & Distributed Learning
- Peter Richtárik
- Bingsheng He
- Qinbin Li
- Jianyu Wang
- Gauri Joshi
- Mehrdad Mahdavi
- Qiang Yang
- Virginia Smith
Citations in the corpus
- Practical and Private (Deep) Learning without Sampling or Shuffling (2021, background): “in the context of distributed settings like federated learning (FL) where one has little control on which subset of the training data one sees at any time (Kairouz et al., 2019; Balle et al., 2020).”
- DISCO: Dynamic and Invariant Sensitive Channel Obfuscation for deep neural networks (2020, background): “For the former, distributed learning techniques such as federated learning [6, 7] and split learning [8, 9] are used, where clients communicate with a centralized server using weights and activations and the latter relies on homomorphic encryption [10, 11] and secure enclaves [12, 13].”
- Privacy Amplification by Decentralization (2020, background): “…n do 4: the same privacy-utility trade-off as a trusted central aggregator that would iteratively aggregate the raw contributions of all users at each round k and perturb the result before sending it to the users, as done in federated learning algorithms with a trusted server (Kairouz et al., 2019).”
- Differentially Private Learning Needs Better Features (or Much More Data) (2020, methodology): “DP has been considered for Federated Learning (Geyer et al., 2017; McMahan et al., 2018), but has also been found to significantly degrade performance in some settings (Yu et al., 2020).”
- Secure Single-Server Aggregation with (Poly)Logarithmic Overhead (2020, methodology): “That paper and subsequent work [7] showed that secure vector summation enables powerful privacy-preserving functionalities such as federated learning [25].”
- Shuffled Model of Federated Learning: Privacy, Communication and Accuracy Trade-offs (2020, methodology): “In this paper we consider a federated learning (FL) framework [KMY + 16, YLCT19, KMA + 19], where the data is generated across m clients.”
- Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning (2020, background): “[13] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.”
- SplitNN-driven Vertical Partitioning (2020, background): “The other variants to SplitNN for distributed private training can also be achieved with Federated Learning (McMahan et al., 2017; Kairouz et al., 2019), but unlike SplitNN it is necessary to share the complete model with all the clients.”
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “Federated learning (FL) is a distributed learning paradigm that aims to train machine learning models from scattered and isolated data [1].”
- Breaking the Communication-Privacy-Accuracy Trilemma (2020, background): “This is even more so in federated learning type settings, where communication occurs over bandwidth-limited wireless links [2].”
- IBM Federated Learning: an Enterprise Framework White Paper V0.1 (2020, background): “[16] provides a good overview of some challenges of federated learning.”
- Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization (2020, methodology): “If we use momentum SGD where the local momentum buffers of active clients are reset to zero at the beginning of each round [25] due to the stateless nature of cross-device FL [2], then we have ai = [1 − ρi , 1 − ρτi−1, .”
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “This leads to an alarming fairness counter-effect with regards to robustness and demonstrates a largely unexplored fairness and robustness trade-off which has already been conjectured in [12].”