Differentially Private Releasing via Deep Generative Model

By S. Ji, Ting Wang. 2018

75 citations recorded by the source corpus.

Privacy-preserving releasing of complex data (e.g., image, text, audio) represents a long-standing challenge for the data mining research community. Due to rich semantics of the data and lack of a priori knowledge about the analysis task, excessive sanitization is often necessary to ensure privacy, leading to significant loss of the data utility. In this paper, we present dp-GAN, a general private releasing framework for semantic-rich data. Instead of sanitizing and then releasing the data, the data curator publishes a deep generative model which is trained using the original data in a differentially private manner; with the generative model, the analyst is able to produce an unlimited amount of synthetic data for arbitrary analysis tasks. In contrast of alternative solutions, dp-GAN highlights a set of key features: (i) it provides theoretical privacy guarantee via enforcing the differential privacy principle; (ii) it retains desirable utility in the released model, enabling a variety of otherwise impossible analyses; and (iii) most importantly, it achieves practical training scalability and stability by employing multi-fold optimization strategies. Through extensive empirical evaluation on benchmark datasets and analyses, we validate the efficacy of dp-GAN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Models for Effective ML on Private, Decentralized Datasets (2019, background): “Prior work has looked at training GANs with DP in a centralized setting (Torkzadehmahani et al., 2019; Xie et al., 2018; Zhang et al., 2018; Frigerio et al., 2019; Beaulieu-Jones et al., 2018; Esteban et al., 2017).”
AdaCliP: Adaptive Clipping for Private SGD (2019, methodology): “At each iteration of training, [37] proposes to use public data to obtain an approximate bound on gradient norm and clip the gradients at this approximate bound.”
Differentially Private Model Publishing for Deep Learning (2019, methodology): “Several existing DP-SGD implementations [1], [6], [20], [45] use moments accountant(MA) method to track privacy loss, while implementing random shuffling or using its APIs to generate data batches which may violate the random sampling assumption required by MA.”
Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference (2018, methodology): “We use a modiﬁed DP-SGD algorithm [2] to ﬁne-tune the entire network, using the following techniques introduced by [70]; warm-starting, where a public dataset is used to initialize the weights of the model, and weights clustering, where the same dataset is used to estimate the gradient l2 norms of…”
Membership Inference Attacks on Machine Learning: A Survey (2021, background): “Many papers [8, 27, 148, 198, 217, 220, 222, 240] have proposed various differentially private generative models to ensure the privacy of their training records.”
When Machine Learning Meets Privacy (2020, unknown): “tive images so that they can preserve privacy of individuals. The authors focus on the generation of synthetic facial images and how they can be used for classification of actual images. Zhang et al. [180] proposed a novel approach based on GNNs to increase privacy of users while releasing semantic ”
Synthetic Data for Deep Learning (2019, methodology): “[696] who used the Improved WGAN framework [217] and obtained excellent results on the synthetic data generated from various subsets of the LSUN dataset [696], which is already a full-scale image dataset, albeit low-resolution (64×64).”