Stealing Machine Learning Models via Prediction APIs

By Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, Thomas Ristenpart. 2016

2,163 citations recorded by the source corpus.

Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.

Subfields: Artificial Intelligence, Computer Networks and Communications, Safety, Robustness & Interpretability, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Bernt Schiele
  3. Mohit Bansal
  4. Yoshua Bengio
  5. Nicholas Carlini
  6. Quoc V. Le
  7. Nicolas Papernot
  8. Dawn Song

In Computer Networks and Communications

  1. Neil Zhenqiang Gong
  2. Binghui Wang
  3. Battista Biggio
  4. Giorgio Giacinto
  5. Prateek Mittal
  6. Jinyuan Jia
  7. Nicholas Carlini
  8. Nicolas Papernot

In Safety, Robustness & Interpretability

  1. Nicolas Papernot
  2. Nicholas Carlini
  3. Dawn Song
  4. Bo Li
  5. Florian Tramèr
  6. Matthias Hein
  7. Bernt Schiele
  8. Prateek Mittal

In Privacy & Security

  1. Peter Kairouz
  2. Vitaly Shmatikov
  3. Dan Boneh
  4. Reza Shokri
  5. Rasmus Pagh
  6. H. Brendan McMahan
  7. Abhradeep Thakurta
  8. Badih Ghazi

Citations in the corpus