Privacy-Preserving Collaborative Deep Learning with Irregular Participants.

By Lingchen Zhao, Yan Zhang, Qian Wang, Yanjiao Chen, Cong Wang, Qin Zou. 2018

10 citations recorded by the source corpus.

With large amounts of data collected from massive sensors, mobile users and institutions becomes widely available, neural network based deep learning is becoming increasingly popular and making great success in many application scenarios, such as image detection, speech recognition and machine translation. While deep learning can provide various benefits, the data for training usually contains highly sensitive information, e.g., personal medical records, and a central location for saving the data may pose a considerable threat to user privacy. In this paper, we present a practical privacy-preserving collaborative deep learning system that allows users (i.e., participants) to cooperatively build a collective deep learning model with data of all participants, without direct data sharing and central data storage. In our system, each participant trains a local model with their own data and only shares model parameters with the others. To further avoid potential privacy leakage from sharing model parameters, we use functional mechanism to perturb the objective function of the neural network in the training process to achieve $\epsilon$-differential privacy. In particular, for the first time, we consider the possibility that the data of certain participants may be of low quality (called \textit{irregular participants}), and propose a solution to reduce the impact of these participants while protecting their privacy. We evaluate the performance of our system on two well-known real-world data sets for regression and classification tasks. The results demonstrate that our system is robust to irregular participants, and can achieve high accuracy close to the centralized model while ensuring rigorous privacy protection.

Subfields: Artificial Intelligence, Health Informatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, methodology): “DPSGD [22] # # MNIST, CIFAR-10 Image Classification w/ DNN DP LSTM [77] # # Reddit Posts Language Model w/ LSTMs DP LOR [78] # # Artificial Data Logistic Regression DP ERM [79] # # Adult, KDD-99 Classification w/ ERM DP GAN [80] # # MNIST, MIMIC-III Data Generation w/ GAN DP GM [81] # # MNIST, CDR, ”
Evaluating Differentially Private Machine Learning in Practice (2019, background): “Finally, high privacy budgets are required for non-convex learning algorithms, such as deep learning [61, 80].”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, unknown): “The utility beneﬁt of using relaxation is evident from the fact that the privacy budget for deep learning algorithms is signiﬁcantly lesser than the prior works of Shokri and Shmatikov [61] and Zhao et al. [81] which do not use any relaxation.”