The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

By Peter Kairouz, Ken Ziyu Liu, Thomas Steinke. 2021

307 citations recorded by the source corpus.

We consider training models on private data that are distributed across user devices. To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server. We present a comprehensive end-toend system, which appropriately discretizes the data and adds discrete Gaussian noise before performing secure aggregation. We provide a novel privacy analysis for sums of discrete Gaussians and carefully analyze the effects of data quantization and modular summation arithmetic. Our theoretical guarantees highlight the complex tension between communication, privacy, and accuracy. Our extensive experimental results demonstrate that our solution is essentially able to match the accuracy to central differential privacy with less than 16 bits of precision per value.

Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Robust Aggregation for Federated Learning (2019, background): “Frameworks to guarantee privacy of user data include differential privacy [31, 45] and homomorphic encryption [35].”
Locally Private k-Means in One Round (2021, background): “This is only for simplicity of analysis; it is possible to split the noise between the users and achieve the same asymptotic guarantees (see Kairouz et al. (2021)).”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, background): “Data aggregation enhances the privacy of federated learning by combining data or gradients from multiple clients, reducing the information from a specific client in the shared information [95].”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, background): “aggregation schemes have been improved in terms of privacy with the addition of DP mechanisms [149, 150, 151, 152] .”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “For more discussion of differential privacy in federated learning, including local differential privacy and other system considerations, we refer readers to Section 5 of Kairouz et al. (2021d) and Section 7.1 of Wang et al. (2021).”