Differentially-Private "Draw and Discard" Machine Learning

By Vasyl Pihur, Aleksandra Korolova, Frederick Liu, Subhash Sankuratripati, M. Yung, Dachuan Huang, Ruogu Zeng. 2018

39 citations recorded by the source corpus.

In this work, we propose a novel framework for privacy-preserving client-distributed machine learning. It is motivated by the desire to achieve differential privacy guarantees in the local model of privacy in a way that satisfies all systems constraints using asynchronous client-server communication and provides attractive model learning properties. We call it "Draw and Discard" because it relies on random sampling of models for load distribution (scalability), which also provides additional server-side privacy protections and improved model quality through averaging. We present the mechanics of client and server components of "Draw and Discard" and demonstrate how the framework can be applied to learning Generalized Linear models. We then analyze the privacy guarantees provided by our approach against several types of adversaries and showcase experimental results that provide evidence for the framework's viability in practical deployments.

Subfields: Artificial Intelligence, Safety Research, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Privacy & Security

Citations in the corpus

Towards Federated Learning at Scale: System Design (2019, background): “In particular, Pihur et al. (2018) proposes an algorithm that learns from users’ data without performing aggregation on the server and with additional formal privacy guarantees.”
LEAF: A Benchmark for Federated Settings (2018, background): “As an example, consider the federated learning paradigm, which focuses on training models directly on federated networks [22, 30, 28].”
Orthogonal Gradient Descent for Continual Learning (2019, background): “through the model prediction gradients not the actual data) is also related differential private learning (Wu et al., 2017; Li et al., 2018; Pihur et al., 2018; Han et al., 2018) and federated learning (Bonawitz et al.”
Privacy-Preserving Bandits (2019, background): “The “Draw and Discard” machine learning system described in (Pihur et al., 2018) is an interesting combination of the approach of bringing the code to the data with differential privacy guarantees.”