PRADA: Protecting Against DNN Model Stealing Attacks

By Mika Juuti, Sebastian Szyller, Alexey Dmitrenko, Samuel Marchal, N. Asokan. 2018

529 citations recorded by the source corpus.

Machine learning (ML) applications are increasingly prevalent. Protecting the confidentiality of ML models becomes paramount for two reasons: (a) a model can be a business advantage to its owner, and (b) an adversary may use a stolen model to find transferable adversarial examples that can evade classification by the original model. Access to the model can be restricted to be only via well-defined prediction APIs. Nevertheless, prediction APIs still provide enough information to allow an adversary to mount model extraction attacks by sending repeated queries via the prediction API. In this paper, we describe new model extraction attacks using novel approaches for generating synthetic queries, and optimizing training hyperparameters. Our attacks outperform state-of-the-art model extraction in terms of transferability of both targeted and non-targeted adversarial examples (up to +29-44 percentage points, pp), and prediction accuracy (up to +46 pp) on two datasets. We provide take-aways on how to perform effective model extraction attacks. We then propose PRADA, the first step towards generic and effective detection of DNN model extraction attacks. It analyzes the distribution of consecutive API queries and raises an alarm when this distribution deviates from benign behavior. We show that PRADA can detect all prior model extraction attacks with no false positives.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

The False Promise of Imitating Proprietary LLMs (2023, background): “There is an existing body of work in this direction, e.g., one can detect whether a particular model is trained via imitation (Krishna et al., 2020; Juuti et al., 2019; Szyller et al., 2019; Maini et al., 2021) or slow model stealing by sacrifing some performance (Wallace et al., 2020; Orekondy et…”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “using model stealing [87, 70, 101, 103, 69, 43]) and have, thus, become standard for evaluation.”
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, background): “Various types of data extraction attacks exist, including model theft attacks [125, 132], gradient leakage [151], and training data extraction attacks [29].”
Stateful Detection of Black-Box Adversarial Attacks (2019, methodology): “There has been related work [23] which explicitly defends against this type of attack.”
Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding (2018, background): “It may be possible to retrieve the model parameters with different model stealing approaches [51], [36], [56], [39], [24].”
Increasing the Cost of Model Extraction with Calibrated Proof of Work (2022, methodology): “Jacobian-TR is a variation of Jacobian proposed by Juuti et al. (2019).”
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020, background): “For example, Juuti et al. (2019) assume adversaries will make consecutive out-of-distribution queries and can thus be detected.”
On the Difficulty of Defending Self-Supervised Learning against Model Extraction (2022, methodology): “Moreover, detection methods like PRADA (Juuti et al., 2019) do not generalize to attacks that utilize only natural data (Orekondy et al.”
Knockoff Nets: Stealing Functionality of Black-Box Models (2018, methodology): “[19, 25] extend the line of work by stealing models (slightly more complex than [35] e.”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “Other works have explored the threats of models themselves like backdoor attack [10, 34, 41, 42] andmodel stealing attack [13, 24, 39, 40, 43].”
A Survey of Privacy Attacks in Machine Learning (2020, methodology): “Several other strategies for selecting the most suitable data for querying the target model use: (i) data that are not synthetic but belong to different domains such as images from different datasets [13, 72], (ii) semi-supervised learning techniques such as rotation loss [107] or MixMatch [5] to au”
Bag of Tricks for Training Data Extraction from Language Models (2023, methodology): “Model extraction attacks and the corresponding protection methods (Tramèr et al., 2016; Juuti et al., 2019; Gong et al., 2020; Wu et al., 2022) focus on the issue of duplicating the functionality of a given model.”
High Accuracy and High Fidelity Extraction of Neural Networks (2019, background): “The other camp, differentiating benign from malicious users, has focused on analyzing query patterns [51, 52].”
Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks (2019, methodology): “Existing defense strategies aim to either detect stealing query patterns (Juuti et al., 2019), or degrade quality of predicted posterior via perturbation .”
DNN Intellectual Property Protection: Taxonomy, Methods, Attack Resistance, and Evaluations (2020, background): “However, the function of the model may also be stolen through model extraction attacks [18, 28].”
Survey of Attacks and Defenses on Edge-Deployed Neural Networks (2019, background): “[52] [53] [59] [60] [61, 62, 63] [65] [71, 72]”
DAWN: Dynamic Adversarial Watermarking of Neural Networks (2019, methodology): “, it may use widely available natural data [33] to mount the attack. Alternatively, it may use only synthetic samples [42] or a mix of a small number of natural samples augmented by synthetic samples [21, 34]. Querying strategy. All model stealing attacks [9, 21, 33, 34, 42] consist of alternating p”
Explainable artificial intelligence: a comprehensive review (2021, background): “The attacks mentioned above and the methods to defend against them have been studied extensively in recent years (Dunn et al. 2020; Juuti et al. 2019).”
Thieves on Sesame Street! Model Extraction of BERT-based APIs (2019, background): “Model extraction attacks have been studied both empirically (Tramèr et al., 2016; Orekondy et al., 2019a; Juuti et al., 2019) and theoretically (Chandrasekaran et al., 2018; Milli et al., 2019), mostly against image classification APIs.”