Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples

By Nicolas Papernot, Patrick Drew McDaniel, Ian J. Goodfellow, Somesh Jha, Z. Berkay Celik, Ananthram Swami. 2016

496 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Red Teaming Language Models with Language Models (2022, methodology): “…it is possible to create adversarial examples for a target model by creating adversarial examples using another model (Szegedy et al., 2014; Liu et al., 2017; Perez et al., 2019), especially when the other model is trained to make similar predictions as the target model (Papernot et al., 2016a,b).”
Adversarial Examples for Evaluating Reading Comprehension Systems (2017, background): “In contrast with prior work in computer vision (Papernot et al., 2017; Narodytska and Kasiviswanathan, 2016; Moosavi-Dezfooli et al., 2017), A DD O NE S ENT does not require any access to the model or to any training data: it generates adversarial examples based solely on the intuition that…”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “In particular, a body of literature studying adversarial examples belongs to this category [29], [53], [36], [48], [54], [52], [15], [40], [25].”
Stealing Machine Learning Models via Prediction APIs (2016, methodology): “[42] make use of improper model extraction techniques in the context”
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks (2018, background): “The first attack subjected to the FAIL analysis is JSMA [35], a well-known targeted evasion attack Transferability of this attack has previously been studied only for an adversary with limited knowledge along the A and I dimensions [37].”
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks (2017, background): “However, two subsequent studies have found that defensive distillation failed to mitigate a variant of JSMA with a division trick [6] and a black-box attack [33].”
On the (Statistical) Detection of Adversarial Examples (2017, methodology): “Further, they proposed a modified version of adversarial training, originally introduced by Goodfellow et al. [11].”
On Detecting Adversarial Perturbations (2017, background): “Moreover, Papernot et al. (2016a) have shown that a potential attacker can construct adversarial examples for a network of unknown architecture by training an auxiliary network on similar data and exploiting the transferability of adversarial inputs.”
Delving into Transferable Adversarial Examples and Black-box Attacks (2016, background): “Further, Papernot et al. (2016a) studied the transferability between deep neural networks and other models such as decision tree, kNN, etc.”
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples (2016, methodology): “We enhance an algorithm introduced in [20] to learn a sub-stitute model for a given classiﬁer simply by querying it for labels on carefully chosen inputs.”
Generating Adversarial Examples with Adversarial Networks (2018, background): “Most of the black-box attack strategies are based on the transferability phenomenon [Papernot et al., 2016], where an adversary can train a local model first and generate adversarial examples against it, hoping the same adversarial examples will also be able to attack the other models.”
The Robust Manifold Defense: Adversarial Training using Generative Models (2017, methodology): “We train our end-to-end substitute network as in the typical black-box setting [36] using input-output pairs of the target model.”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “To make deep neural networks more robust to adversarial attacks, several defensive algorithms have been proposed recently [16, 27, 12, 11, 26].”