Differentially Private Generative Adversarial Network

By Liyang Xie, Kaixiang Lin, Fei Wang, Jiayu Zhou. 2018

593 citations recorded by the source corpus.

Generative Adversarial Network (GAN) and its variants have recently attracted intensive research interests due to their elegant theoretical foundation and excellent empirical performance as generative models. These tools provide a promising direction in the studies where data availability is limited. One common issue in GANs is that the density of the learned generative distribution could concentrate on the training data points, meaning that they can easily remember training samples due to the high model complexity of deep networks. This becomes a major concern when GANs are applied to private or sensitive data such as patient medical records, and the concentration of distribution may divulge critical patient information. To address this issue, in this paper we propose a differentially private GAN (DPGAN) model, in which we achieve differential privacy in GANs by adding carefully designed noise to gradients during the learning procedure. We provide rigorous proof for the privacy guarantee, as well as comprehensive empirical evidence to support our analysis, where we demonstrate that our method can generate high quality data points at a reasonable privacy level.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, methodology): “MA is also employed by the works [87, 88, 89, 80, 81, 90] all of which use perturbed gradients.”
Generative Models for Effective ML on Private, Decentralized Datasets (2019, background): “Prior work has looked at training GANs with DP in a centralized setting (Torkzadehmahani et al., 2019; Xie et al., 2018; Zhang et al., 2018; Frigerio et al., 2019; Beaulieu-Jones et al., 2018; Esteban et al., 2017).”
Synthetic Data for Deep Learning (2019, methodology): “[657] present the differentially private GAN (DPGAN) model, which is basically the already classical Wasserstein GAN [26, 217] but with additional noise on the gradient of the Wasserstein distance, in a fashion following the DP-SGD approach (Section 7.”
Generation and evaluation of synthetic patient data (2020, background): “An interesting direction of research has been in converting popular machine learning algorithms, such as deep learning algorithms, to differentially private algorithms via techniques such as gradient clipping and noise addition [45, 46].”
Private FL-GAN: Differential Privacy Synthetic Data Generation Based on Federated Learning (2020, methodology): “[11, 18] Given the sampling probability q = m/M (M represents the dataset size), the number of discriminator iterations in each inner loop T d and privacy violation δ , for any positive (cid:15) , the parameters of discriminator guarantee ( (cid:15), δ ) -differential privacy with respect to all…”
Synthesis of Realistic ECG using Generative Adversarial Networks (2019, unknown): “[37].”
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models (2019, methodology): “ designed to protect the membership privacy of individual samples and is by constructing a defense mechanism against MIA. Recent works propose to train GAN models with differential privacy constraint [2, 7, 11, 35, 66, 73] and publicize the DP-trained models instead of the raw data, which allows sha”
Security and Privacy Issues in Deep Learning (2018, background): “[94] proposed a Differentially Private Generative Adversarial Network (DPGAN).”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, methodology): “For example, GAN-based methods were explored in DP-GAN (Xie et al., 2018) and PATE-GAN (Yoon et al.”
DP-CGAN: Differentially Private Synthetic Data and Label Generation (2019, methodology): “Finally, in DPGAN frameworks [43, 18] the discriminator gradients are clipped and perturbed by adding Gaussian noise to gradients of the discriminator loss, while in our framework, Gaussian noise is added to the accumulation of clipped gradients of discriminator loss on real data and clipped gradien”
Membership Inference Attacks on Machine Learning: A Survey (2021, background): “Many papers [8, 27, 148, 198, 217, 220, 222, 240] have proposed various differentially private generative models to ensure the privacy of their training records.”
Harnessing the power of synthetic data in healthcare: innovation, application, and privacy (2023, background): “PATE-GAN has proven superior to conventional, yet outdated benchmarks, such as Differential Private-GAN (DP-GAN) 71 and excels in generating synthetic data that closely resemble the original dataset.”
Generative Adversarial Networks in Time Series: A Systematic Literature Review (2022, background): “Xie et al. Xie2018 proposed a differentially private GAN (DPGAN) that achieved DP by adding noise gradients to the optimizer during the training phase [103].”
Machine Learning for Synthetic Data Generation: a Review (2023, methodology): “Taking advantage of the GAN, several methods are proposed to generate synthetic data to get better effect [49], [167]–[169] which closely matches the distribution of the source data than the hidden Markov model-based approach [71], RBF based approach [170] and Auto-encoder based approach [14].”
Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Data (2019, methodology): “[19] developed a dp-GAN based on the dp-SGD providing some optimizations in order to improve performances focusing on the generation of images.”
PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees (2018, methodology): “The most related previous work to this paper is DPGAN [32].”