Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning
By Tribhuvanesh Orekondy, Seong Joon Oh, Yang Zhang, Bernt Schiele, Mario Fritz. 2018
43 citations recorded by the source corpus.
Federated Learning (FL) systems are gaining popularity as a solution to training Machine Learning (ML) models from large-scale user data collected on personal devices (e.g., smartphones) without their raw data leaving the device. At the core of FL is a network of anonymous user devices sharing minimal training information (model parameter deltas) computed locally on personal data. However, the degree to which user-specific information is encoded in the model deltas is poorly understood. In this paper, we identify model deltas encode subtle variations in which users capture and generate data. The variations provide a powerful statistical signal, allowing an adversary to effectively deanonymize participating devices using a limited set of auxiliary data. We analyze resulting deanonymization attacks on diverse tasks on real-world (anonymized) user-generated data across a range of closed- and open-world scenarios. We study various strategies to mitigate the risks of deanonymization. As random perturbation methods do not offer convincing operating points, we propose data-augmentation strategies which introduces adversarial biases in device data and thereby, offer substantial protection against deanonymization threats with little effect on utility.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Chaoyang He
- Yang Liu
- Murali Annavaram
- Xuefei Yin
- Qiang Yang
- Jinhyun So
- Abhishek Singh
- Praneeth Vepakomma
Citations in the corpus
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “…robust [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 22], and proposing methods such as differential privacy (DP) and secure multiparty computation (SMPC) to protect privacy [25, 26, 27, 28, 29, 30, 31, 32, 33] (see Table 8 in the appendix for a taxonomy of the federated learning literature).”
- A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “Therefore, they (weight update, gradient update and the final model) contain sensitive information of local data [133, 142, 167].”