Differentially Private Data Generative Models

By Qingrong Chen, Chong Xiang, Minhui Xue, Bo Li, Nikita Borisov, Dali Kaafar, Haojin Zhu. 2018

86 citations recorded by the source corpus.

Deep neural networks (DNNs) have recently been widely adopted in various applications, and such success is largely due to a combination of algorithmic breakthroughs, computation resource improvements, and access to a large amount of data. However, the large-scale data collections required for deep learning often contain sensitive information, therefore raising many privacy concerns. Prior research has shown several successful attacks in inferring sensitive training data information, such as model inversion, membership inference, and generative adversarial networks (GAN) based leakage attacks against collaborative deep learning. In this paper, to enable learning efficiency as well as to generate data with privacy guarantees and high utility, we propose a differentially private autoencoder-based generative model (DP-AuGM) and a differentially private variational autoencoder-based generative model (DP-VaeGM). We evaluate the robustness of two proposed models. We show that DP-AuGM can effectively defend against the model inversion, membership inference, and GAN-based attacks. We also show that DP-VaeGM is robust against the membership inference attack. We conjecture that the key to defend against the model inversion and GAN-based attacks is not due to differential privacy but the perturbation of training data. Finally, we demonstrate that both DP-AuGM and DP-VaeGM can be easily integrated with real-world machine learning applications, such as machine learning as a service and federated learning, which are otherwise threatened by the membership inference attack and the GAN-based attack, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

This Face Does Not Exist... But It Might Be Yours! Identity Leakage in Generative Models (2020, background): “Similarly, Chen et al.[5] offer GANs based on auto-encoders and variational autoencoders designed to protect privacy while retaining high data utility.”
Membership Inference Attacks on Machine Learning: A Survey (2021, background): “Many papers [21, 27, 30, 60, 76, 77, 82, 83, 86, 107, 110, 145, 162, 179, 199, 217, 227, 230] have applied DP to ML models to mitigate MIAs.”
ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models (2018, background): “Besides the above, many recent works have tackled security and privacy in machine learning from various perspectives [39], [16], [9], [37], [7], [27], [20], [45], [36], [21], [40], [29], [41], [20].”
Machine Learning for Synthetic Data Generation: a Review (2023, background): “…GAN CNN image [176] differential privacy Bayesian network NA (w.o. DNN) attribute [177] differential privacy statistical database NA (w.o. DNN) attribute [178] differential privacy Graduate Update Method NA (w.o. DNN) attribute [179] differential privacy autoencoder autoencoder image and attribute”
DP-CGAN: Differentially Private Synthetic Data and Label Generation (2019, background): “However, in spite of its popularity, there have been very few recent studies on training GANs in a differentially private way [43, 18, 44, 7, 8].”