Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning

By Nhathai Phan, Xintao Wu, Han Hu, Dejing Dou. 2017

222 citations recorded by the source corpus.

In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks. To achieve this, we figure out a way to perturb affine transformations of neurons, and loss functions used in deep neural networks. In addition, our mechanism intentionally adds "more noise" into features which are "less relevant" to the model output, and vice-versa. Our theoretical analysis further derives the sensitivities and error bounds of our mechanism. Rigorous experiments conducted on MNIST and CIFAR-10 datasets show that our mechanism is highly effective and outperforms existing solutions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, methodology): “One solution is replacing the non-convex function with an approximate convex polynomial function [82, 83, 84] and then using objective function perturbation.”
Differentially Private Generative Adversarial Network (2018, methodology): “…work [35], and Differentially Private Learning in Neural Network. several literatures: Abadi et al. [1] studied a gradient clipping method et al. [25] combined Laplacian mechanism with machine teaching Phan et al. [27] developed “adaptive Laplace Mechanism” Phan et al. [26] developed a private”
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges (2020, background): “Laplace mechanism adds noise of Laplace distribution [109].”
Security and Privacy Issues in Deep Learning (2018, background): “[98] developed a novel mechanism, called Adaptive Laplace Mechanism (AdLM).”
When Machine Learning Meets Privacy (2020, background): “One possible solution is to use a convex approximation of the loss function [124].”
Secure and Robust Machine Learning for Healthcare: A Survey (2020, background): “Various approaches for differential privacy have been proposed in the literature, e.g., private aggregation of teacher ensembles (PATE) for private ML [156], differentially private stochastic gradient descent (DP-SGD) algorithm [154], moments accountant [157], hyperparameter selection [158], Laplace”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “In this regard, different techniques can be use for preserving privacy, e.g., using cryptographic approaches (like homomorphic encryption [74], and multi-party computation [75], etc.), differential privacy [76], and federated learning [77].”