High Accuracy and High Fidelity Extraction of Neural Networks

By Matthew Jagielski, Nicholas Carlini, David Berthelot, Alexey Kurakin, Nicolas Papernot. USENIX Security Symposium, 2019

476 citations recorded by the source corpus.

In a model extraction attack, an adversary steals a copy of a remotely deployed machine learning model, given oracle prediction access. We taxonomize model extraction attacks around two objectives: *accuracy*, i.e., performing well on the underlying learning task, and *fidelity*, i.e., matching the predictions of the remote victim classifier on any input. To extract a high-accuracy model, we develop a learning-based attack exploiting the victim to supervise the training of an extracted model. Through analytical and empirical arguments, we then explain the inherent limitations that prevent any learning-based strategy from extracting a truly high-fidelity model---i.e., extracting a functionally-equivalent model whose predictions are identical to those of the victim model on all possible inputs. Addressing these limitations, we expand on prior work to develop the first practical functionally-equivalent extraction attack for direct extraction (i.e., without training) of a model's weights. We perform experiments both on academic datasets and a state-of-the-art image classifier trained with 1 billion proprietary images. In addition to broadening the scope of model extraction research, our work demonstrates the practicality of model extraction attacks against production-grade systems.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/52a222d38a8640499010d470d5589a81882bc425

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Thieves on Sesame Street! Model Extraction of BERT-based APIs (2019, result): “…models contrasts with prior work, where large-scale attacks require at minimum that the adversary can access a small amount of semantically-coherent data relevant to the task (Papernot et al., 2017; Correia-Silva et al., 2018; Orekondy et al., 2019a; Pal et al., 2019; Jagielski et al., 2019).”
A Systematic Review on Model Watermarking for Neural Networks (2020, methodology): “A task accuracy extraction aims at extracting a model that solves approximately the same underlying decision task [Jagielski et al., 2020].”
On the Difficulty of Defending Self-Supervised Learning against Model Extraction (2022, methodology): “The attacker uses the query-output pairs to rapidly train a local model, often at a fraction of the cost of the victim model (Jagielski et al., 2020).”
Dataset Inference: Ownership Resolution in Machine Learning (2021, background): “Model extraction (Tram`er et al., 2016; Jagielski et al., 2020; Truong et al., 2021) is the process where an adversary tries to steal a copy of a machine learning model, that may have been remotely deployed (such as over a prediction API).”
A Survey of Privacy Attacks in Machine Learning (2020, methodology): “Several other strategies for selecting the most suitable data for querying the target model use: (i) data that are not synthetic but belong to different domains such as images from different datasets [13, 72], (ii) semi-supervised learning techniques such as rotation loss [107] or MixMatch [5] to au”
Increasing the Cost of Model Extraction with Calibrated Proof of Work (2022, methodology): “This model extraction method shows that is is possible to steal a model even with a small number of labeled examples (Jagielski et al., 2020).”