Differential Privacy Preservation for Deep Auto-Encoders: an Application of Human Behavior Prediction

By Nhathai Phan, Yue Wang, Xintao Wu, Dejing Dou. 2016

252 citations recorded by the source corpus.

In recent years, deep learning has spread beyond both academia and industry with many exciting real-world applications. The development of deep learning has presented obvious privacy issues. However, there has been lack of scientific study about privacy preservation in deep learning. In this paper, we concentrate on the auto-encoder, a fundamental component in deep learning, and propose the deep private auto-encoder (dPA). Our main idea is to enforce ε-differential privacy by perturbing the objective functions of the traditional deep auto-encoder, rather than its results. We apply the dPA to human behavior prediction in a health social network. Theoretical analysis and thorough experimental evaluations show that the dPA is highly effective and efficient, and it significantly outperforms existing solutions.

Subfields: Artificial Intelligence, Safety Research, Generative Models, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Generative Models

In Privacy & Security

Citations in the corpus

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, methodology): “DP was used in deep auto-encoders in [67].”
Privacy in Deep Learning: A Survey (2020, methodology): “One solution is replacing the non-convex function with an approximate convex polynomial function [82, 83, 84] and then using objective function perturbation.”
Differentially Private Releasing via Deep Generative Model (2018, background): “[27], proposed to apply the functional mechanism [40] to train differentially private auto-encoders.”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, unknown): “An alternative approach is to replace the non-convex function with a convex polynomial function [56,57], and then use the standard objective perturbation.”
Optimal Noise-Adding Mechanism in Additive Differential Privacy (2018, background): “…learning (Jain et al., 2012), statistical risk minimization (Duchi et al., 2012), deep learning (Shokri and Shmatikov, 2015; Abadi et al., 2016; Phan et al., 2016; Agarwal et al., 2018), hypothesis testing (Sheffet, 2018), matrix completion (Jain et al., 2018), expectation maximization (Park…”
Interpretable and Differentially Private Predictions (2019, background): “An alternative approach is to directly perturb the objective with additive noise (Zhang et al. 2012; Phan et al. 2016; Phan, Wu, and Dou 2017).”
Security and Privacy Issues in Deep Learning (2018, background): “[95] proposed deep private auto-encoder (dPA) and proved that the dPA is differential private based on the functional mechanism.”
Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence (2020, background): “To alleviate the privacy leakage, Phan et al. [38] achieve learning autoencoder with DP by perturbing loss function.”
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning (2017, methodology): “Given a fully-connected layer, as in [10], given an input xi, each perturbed neuron hxi(W ) can be directly normalized as follows: hxi ← ( hxi(W )−χ ) /(φ−χ),”
Preserving differential privacy in convolutional deep belief networks (2017, methodology): “They are either designed for a speciﬁc deep learning model, i.e., deep auto-encoders [61], or they are aﬀected by the number of training epochs [4, 67].”
Locally Differentially Private Data Collection and Analysis (2019, background): “One of the many topics in DP research is differentially private empirical risk minimization for machine learning [18, 10], especially for deep neural networks [1, 40, 59, 2, 53].”