Manipulation Attacks in Local Differential Privacy
By Albert Cheu, Adam D. Smith, Jonathan Ullman. 2019
124 citations recorded by the source corpus.
Local differential privacy is a widely studied restriction on distributed algorithms that collect aggregates about sensitive user data, and is now deployed in several large systems. We initiate a systematic study of a fundamental limitation of locally differentially private protocols: they are highly vulnerable to adversarial manipulation. While any algorithm can be manipulated by adversaries who lie about their inputs, we show that any noninteractive locally differentially private protocol can be manipulated to a much greater extent—when the privacy level is high, or the domain size is large, a small fraction of users in the protocol can completely obscure the distribution of the honest users’ input. We also construct protocols that are optimally robust to manipulation for a variety of common tasks in local differential privacy. Finally, we give simple experiments validating our theoretical results, and demonstrating that proto-cols that are optimal without manipulation can have dramatically different levels of robustness to manipulation. Our results suggest caution when deploying local differential privacy and reinforce the importance of efficient cryptographic techniques for the distributed emulation of centrally differentially private mechanisms.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Albert Cheu
- Borja Balle
- Badih Ghazi
- James Bell
- Peter Kairouz
- Adrià Gascón
- Kobbi Nissim
- Maxim Zhilyaev
Citations in the corpus
- Private Summation in the Multi-Message Shuffle Model (2020, background): “[8], which shows that in the local model of differential privacy, a dishonest user’s effect on the outcome is of magnitude O(1/ ).”
- Differentially Private Histograms in the Shuffle Model from Fake Users (2021, methodology): “We would like to thank Kobbi Nissim, Rasmus Pagh, and Jonathan Ullman for discussion and insight for the count-min analysis.”
- FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping (2020, unknown): “[17], [25], [45], neural networks [11], [18], [27], [30], [36], [37], and graph-based methods [20], [41], [49], as well as distributed privacy-preserving data analytics [10], [14].”