Preserving differential privacy in convolutional deep belief networks

By Nhathai Phan, Xintao Wu, Dejing Dou. 2017

88 citations recorded by the source corpus.

The remarkable development of deep learning in medicine and healthcare domain presents obvious privacy issues, when deep neural networks are built on users’ personal and highly sensitive data, e.g., clinical records, user profiles, biomedical images, etc. However, only a few scientific studies on preserving privacy in deep learning have been conducted. In this paper, we focus on developing a private convolutional deep belief network (pCDBN), which essentially is a convolutional deep belief network (CDBN) under differential privacy. Our main idea of enforcing ϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document}-differential privacy is to leverage the functional mechanism to perturb the energy-based objective functions of traditional CDBNs, rather than their results. One key contribution of this work is that we propose the use of Chebyshev expansion to derive the approximate polynomial representation of objective functions. Our theoretical analysis shows that we can further derive the sensitivity and error bounds of the approximate polynomial representation. As a result, preserving differential privacy in CDBNs is feasible. We applied our model in a health social network, i.e., YesiWell data, and in a handwriting digit dataset, i.e., MNIST data, for human behavior prediction, human behavior classification, and handwriting digit recognition tasks. Theoretical analysis and rigorous experimental evaluations show that the pCDBN is highly effective. It significantly outperforms existing solutions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, methodology): “One solution is replacing the non-convex function with an approximate convex polynomial function [82, 83, 84] and then using objective function perturbation.”
Interpretable and Differentially Private Predictions (2019, methodology): “An alternative approach is to directly perturb the objective with additive noise (Zhang et al. 2012; Phan et al. 2016; Phan, Wu, and Dou 2017).”
Differentially Private Generative Adversarial Network (2018, methodology): “…work [35], and Differentially Private Learning in Neural Network. several literatures: Abadi et al. [1] studied a gradient clipping method et al. [25] combined Laplacian mechanism with machine teaching Phan et al. [27] developed “adaptive Laplace Mechanism” Phan et al. [26] developed a private”
Security and Privacy Issues in Deep Learning (2018, methodology): “[96] introduced the private convolutional deep belief network (pCDBN), and they utilized the Chebyshev expansion to approximate the objective function to the polynomial form.”
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy (2023, background): “Energy-based Yes While such losses are common for convolutional deep-belief networks, this loss makes it hard to reason about global sensitivity (Phan et al., 2017).”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, methodology): “[56, 57] Deep Learning Objective YesiWell 254 30 2 1”
Evaluating Differentially Private Machine Learning in Practice (2019, methodology): “An alternative approach is to replace the non-convex function with a convex polynomial function [56, 57], and then use the standard objective perturbation.”