Differentially Private Federated Learning: A Client Level Perspective
By Robin C. Geyer, Tassilo Johannes Klein, Moin Nabi. 2017
1,680 citations recorded by the source corpus.
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance.
Subfields: Artificial Intelligence, Computer Science Applications, Federated & Distributed Learning, Privacy & Security
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Science Applications
- Ling Liu
- Jakub Konecný
- Carole-Jean Wu
- Young Geun Kim
- Dusist Niyato
- H. Vincent Poor
- Yaochu Jin
- Hadi Esmaeilzadeh
In Federated & Distributed Learning
- Qiang Yang
- Virginia Smith
- Chaoyang He
- Peter Richtárik
- Jakub Konecný
- Murali Annavaram
- Tian Li
- Yang Liu
In Privacy & Security
- Qiang Yang
- Lingjuan Lyu
- Vitaly Shmatikov
- Amir Salman Avestimehr
- Ramesh Raskar
- Praneeth Vepakomma
- Congzheng Song
- Mario Fritz
Citations in the corpus
- How To Backdoor Federated Learning (2018, background): “Alternative aggregationmechanisms [13, 20, 68, 71], such as coordinate-wise or geometric medians, greatly reduce the accuracy of complex models on non-i.”
- Ensemble Distillation for Robust Model Fusion in Federated Learning (2020, background): “These include adding differential privacy (Geyer et al., 2017) for client models, or performing hierarchical and decentralized model fusion through synchronizing locally inferred logits e.”
- Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, methodology): “Compared with conventional works, such as [34], [35], which focus mainly on simulation results, our theoretical performance analysis is more efficient for finding the optimal parameters, e.”
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “…robust [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 22], and proposing methods such as differential privacy (DP) and secure multiparty computation (SMPC) to protect privacy [25, 26, 27, 28, 29, 30, 31, 32, 33] (see Table 8 in the appendix for a taxonomy of the federated learning literature).”
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “(v) weak differential private (DP) defense [27, 86] where a Gaussian noise with small standard deviations (σ) is added to the aggregated global model.”
- Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy (2020, methodology): “badi et al. [27]). On the other hand, in the case of a malicious or compromised server, or in the case of potential eavesdropping, DP can be applied on the local clients3 for protecting their privacy [28,29,30]. 5.2. The Machine Learning paradigm in a federated setting In the following, we describe ”
- Privacy in Deep Learning: A Survey (2020, methodology): “w/ DNN Distributed DP [87] # # eICU, TCGA Classification w/ DNN DP model publishing [88] # # MNIST, CIFAR Image Classification w/ DNN DP federated learning [89] # # MNIST Image Classification w/ DNN ScalarDP, PrivUnit [90] # # MNIST, CIFAR Image Classification w/ DNN DSSGD [21] # # MNIST, SVHN Image”
- Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning (2020, background): “from the local datasets cannot be identified from the local updates sent to the server, by adding artificial noise to the local updates at the clients’ side [9], [28], [29].”
- Differentially Private Meta-Learning (2019, background): “, 2017), have inspired many works also concerning DP in a multi-user setup (Agarwal et al., 2018; Duchi et al.; Geyer et al., 2018; McMahan et al., 2018; Truex et al., 2019), we are the first to consider task-global DP as a useful variation on standard DP settings.”
- Gradient Descent with Compressed Iterates (2019, background): “This situation is common in large-scale distributed optimization involving many edge devices, and common challenges include data heterogeneity [37], privacy [7], resource management [25, 33], and system heterogeneity as well as communication efficiency [16, 18].”
- On the Convergence of FedAvg on Non-IID Data (2019, background): “The research work on FL is focused on the communication-efficiency [10, 16, 21, 27] and data privacy [1, 2, 4, 5, 17].”
- Differential Privacy Has Disparate Impact on Model Accuracy (2019, methodology): “Some of our experiments involve federated learning [16, 25, 26].”
- Federated Heavy Hitters Discovery with Differential Privacy (2019, background): “…Xu et al., 2016; Zhou and Lin, 2018; Kairouz et al., 2016; Wang et al., 2017; Bassily et al., 2017; Acharya et al., 2018; Ye and Barg, 2018; Avent et al., 2017; Bun et al., 2018; Cormode et al., 2018), and some recent works combine FL with central DP (Geyer et al., 2017; McMahan et al., 2018).”
- Federated Machine Learning (2019, background): “statistical challenges [60, 77] and improving security [9, 23] in federated learning.”
- LEAF: A Benchmark for Federated Settings (2018, background): “…used and yet do not provide a realistic model of a federated scenario, e.g., artificial partitions of MNIST, MNIST-fashion or CIFAR-10 [22, 13, 8, 3, 11, 32, 34]; (2) realistic but proprietary federated datasets, e.g., data from an unnamed social network in [22], crowdsourced voice commands…”