Privacy-preserving deep learning

By Reza Shokri, Vitaly Shmatikov. 2015

2,700 citations recorded by the source corpus.

Deep learning based on artificial neural networks is a very popular approach to modeling, classifying, and recognizing complex data such as images, speech, and text. The unprecedented accuracy of deep learning methods has turned them into the foundation of new AI-based services on the Internet. Commercial companies that collect user data on a large scale have been the main beneficiaries of this trend since the success of deep learning techniques is directly proportional to the amount of data available for training. Massive data collection required for deep learning presents obvious privacy issues. Users' personal, highly sensitive data such as photos and voice recordings is kept indefinitely by the companies that collect it. Users can neither delete it, nor restrict the purposes for which it is used. Furthermore, centrally kept data is subject to legal subpoenas and extrajudicial surveillance. Many data owners-for example, medical institutions that may want to apply deep learning methods to clinical records-are prevented by privacy and confidentiality concerns from sharing the data and thus benefitting from large-scale deep learning. In this paper, we present a practical system that enables multiple parties to jointly learn an accurate neural-network model for a given objective without sharing their input datasets. We exploit the fact that the optimization algorithms used in modern deep learning, namely, those based on stochastic gradient descent, can be parallelized and executed asynchronously. Our system lets participants train independently on their own datasets and selectively share small subsets of their models' key parameters during training. This offers an attractive point in the utility/privacy tradeoff space: participants preserve the privacy of their respective data while still benefitting from other participants' models and thus boosting their learning accuracy beyond what is achievable solely on their own inputs. We demonstrate the accuracy of our privacy-preserving deep learning on benchmark datasets.

Subfields: Artificial Intelligence, Health Informatics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Qiang Yang
  3. Mohit Bansal
  4. Yoshua Bengio
  5. Bernt Schiele
  6. Quoc V. Le
  7. Dacheng Tao
  8. Philip S. Yu

In Health Informatics

  1. Lingjuan Lyu
  2. Qiang Yang
  3. Congzheng Song
  4. Amir Houmansadr
  5. Hangyu Zhu
  6. Yaochu Jin
  7. David E. Evans
  8. Bargav Jayaraman

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Nicolas Papernot
  3. Florian Tramèr
  4. Reza Shokri
  5. Dawn Song
  6. Yang Zhang
  7. Prateek Mittal
  8. Vitaly Shmatikov

Citations in the corpus