Federated Learning with Bayesian Differential Privacy

By Aleksei Triastcyn, Boi Faltings. 2019

212 citations recorded by the source corpus.

We consider the problem of reinforcing federated learning with formal privacy guarantees. We propose to employ Bayesian differential privacy, a relaxation of differential privacy for similarly distributed data, to provide sharper privacy loss bounds. We adapt the Bayesian privacy accounting method to the federated setting and suggest multiple improvements for more efficient privacy budgeting at different levels. Our experiments show significant advantage over the state-of-the-art differential privacy bounds for federated learning on image classification tasks, including a medical application, bringing the privacy budget below ε = 1 at the client level, and below ε = 0.1 at the instance level. Lower amounts of noise also benefit the model accuracy and reduce the number of communication rounds.

Subfields: Artificial Intelligence, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Murali Annavaram
3. Chaoyang He
4. Yang Liu
5. S. Avestimehr
6. Jiankun Hu
7. Yanming Zhu
8. Xuefei Yin

In Sociology and Political Science

1. Zaid Al-Ars
2. Jun Zhao
3. Peilin Zhao
4. Ramesh Raskar
5. Qiang Yang
6. David Enthoven
7. Dusist Niyato
8. Jiankun Hu

Citations in the corpus

• FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “…robust [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 22], and proposing methods such as differential privacy (DP) and secure multiparty computation (SMPC) to protect privacy [25, 26, 27, 28, 29, 30, 31, 32, 33] (see Table 8 in the appendix for a taxonomy of the federated learning literature).”
• Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “Triastcyn and Faltings [28] introduce Bayesian differential privacy, a natural relaxation of differential privacy that provides better privacy guarantees for clients.”
• Local Differential Privacy-Based Federated Learning for Internet of Things (2020, background): “Triastcyn and Faltings [39] employ Bayesian differential privacy on FL.”
• Differential Privacy for Deep and Federated Learning: A Survey (2022, methodology): “In addition, the authors are based on Bayesian DP [129] to 1) protect the privacy of clients by adding Gaussian noise to gradient depending on the data distribution, and 2) to track the privacy leakage due to synchronization rounds of FL.”
• Federated Learning for Healthcare: Systematic Review and Architecture Proposal (2022, methodology): “Triastcyn and Faltings [79] protect the parameters from local models with the Bayesian differential privacy technique.”
• A Comprehensive Survey of Privacy-preserving Federated Learning (2021, methodology): “WHO RA1 [61, 196, 228] [26, 137, 218] [157, 169, 198] [144, 164, 199] [63, 70, 193] [16, 174, 191] [78, 110, 228] [52, 81, 216] [32, 56, 222] [30, 127, 173] RA2 [114, 175, 187] [215, 221, 224] [29, 111, 113] [13, 199] [3, 10, 40] [109, 155, 168] [45, 172, 193] [20, 52, 57] [56, 165, 197] [173, 199, ”
• An Overview of Federated Deep Learning Privacy Attacks and Defensive Strategies (2020, background): “[55] demonstrated FL with Bayesian DP which may provide faster convergence in”