LDP-Fed: federated learning with local differential privacy

By Stacey Truex, Ling Liu, Ka-Ho Chow, Mehmet Emre Gursoy, Wenqi Wei. 2020

534 citations recorded by the source corpus.

This paper presents LDP-Fed, a novel federated learning system with a formal privacy guarantee using local differential privacy (LDP). Existing LDP protocols are developed primarily to ensure data privacy in the collection of single numerical or categorical values, such as click count in Web access logs. However, in federated learning model parameter updates are collected iteratively from each participant and consist of high dimensional, continuous values with high precision (10s of digits after the decimal point), making existing LDP protocols inapplicable. To address this challenge in LDP-Fed, we design and develop two novel approaches. First, LDP-Fed's LDP Module provides a formal differential privacy guarantee for the repeated collection of model training parameters in the federated training of large-scale neural networks over multiple individual participants' private datasets. Second, LDP-Fed implements a suite of selection and filtering techniques for perturbing and sharing select parameter updates with the parameter server. We validate our system deployed with a condensed LDP protocol in training deep neural networks on public data. We compare this version of LDP-Fed, coined CLDP-Fed, with other state-of-the-art approaches with respect to model accuracy, privacy preservation, and system capabilities.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lichao Sun
2. Lingjuan Lyu
3. Philip S. Yu
4. Qiang Yang
5. Dacheng Tao
6. Bo Du
7. Jiankun Hu
8. Yanming Zhu

Citations in the corpus

• Federated Model Distillation with Noise-Free Differential Privacy (2020, methodology): “A recent work (Truex et al. 2020) utilized Condensed Local Differential Privacy (αCLDP) into federated learning.”
• LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy (2020, background): “Last, the most recent work [Truex et al., 2020] utilizes Condensed Local Differential Privacy (α-CLDP) into federated learning, with 86.93% accuracy on FMNIST dataset.”
• A Framework for Evaluating Gradient Leakage Attacks in Federated Learning (2020, background): “Membership attacks [34,29,36] exploit the statistical differences between the model prediction on its training set and the prediction on unseen data to infer the membership of training data with high confidence.”
• Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, background): “porary work called LDP-FL [86] achieves better performance on both effectiveness and efficiency than [87] with the special communication design for deep learning approaches.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “DP hides real original information by adding noise to the data or gradients before sending them to the server, including local DP [204], hybrid DP [135], shuffle model [61], etc.”
• A Comprehensive Survey of Privacy-preserving Federated Learning (2021, methodology): “WHO RA1 [61, 196, 228] [26, 137, 218] [157, 169, 198] [144, 164, 199] [63, 70, 193] [16, 174, 191] [78, 110, 228] [52, 81, 216] [32, 56, 222] [30, 127, 173] RA2 [114, 175, 187] [215, 221, 224] [29, 111, 113] [13, 199] [3, 10, 40] [109, 155, 168] [45, 172, 193] [20, 52, 57] [56, 165, 197] [173, 199, ”