Query-Efficient Black-box Adversarial Examples

By Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin. 2017

53 citations recorded by the source corpus.

Current neural network-based image classifiers are susceptible to adversarial examples, even in the black-box setting, where the attacker is limited to query access without access to gradients. Previous methods --- substitute networks and coordinate-based finite-difference methods --- are either unreliable or query-inefficient, making these methods impractical for certain problems. We introduce a new method for reliably generating adversarial examples under more restricted, practical black-box threat models. First, we apply natural evolution strategies to perform black-box attacks using two to three orders of magnitude fewer queries than previous methods. Second, we introduce a new algorithm to perform targeted adversarial attacks in the partial-information setting, where the attacker only has access to a limited number of target classes. Using these techniques, we successfully perform the first targeted adversarial attack against a commercially deployed machine learning system, the Google Cloud Vision API, in the partial information setting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Online Adversarial Attacks (2021, background): “(2017), where the attacker has complete access, and the less permissive blackbox threat model where an attacker only has partial information, like the ability to query the target model (Chen et al., 2017; Ilyas et al., 2019; Papernot et al., 2016).”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “Recently, there has been significant interest (Papernot et al., 2017; Chen et al., 2017; Bhagoji et al., 2017; Ilyas et al., 2017) in designing black-box attacks for neural network models.”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, methodology): “In initial experiments, we also tried natural evolutionary strategies (Wierstra et al., 2008; Ilyas et al., 2017) and zeroorder optimization (Chen et al.”
Adversarial Training Versus Weight Decay (2018, methodology): “The technique was shown to not hold in a more general black-box threat model using a query-efﬁcient gradient estimator (Ilyas et al., 2017; Ilyas, 2018), and a gradient-free genetic algorithm (Alzantot et al., 2018). therefore traditional regularization methods are not sufﬁcient to mitigate…”
Robust Perception through Analysis by Synthesis (2018, methodology): “We tried to run each gradient-based attack in two versions: one with the real gradient of the model (if available), one with gradients estimated according to [Ilyas et al., 2017a].”
Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach (2018, methodology): “[13] also solves a score-based attack problem using an evolutionary algorithm and it shows their method could be applied to hard-label black-box setting as well.”
Adversarially Robust Neural Architectures (2020, unknown): “Adversarial attacks are generally divided into two groups, the white-box case [9], [23], [24], [25] and black-box case [26], [27], [28], [29].”
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text (2018, background): “Just as later work in the space of images showed black-box attacks are possible [22, 35]; we expect that our attacks can be extended to black-box attacks.”
Adversarial Robustness Toolbox v1.0.0 (2018, methodology): “The QueryEfficientBBGradientEstimation class implements the efﬁcient algorithm proposed in [29].”
Evaluation and Design of Robust Neural Network Defenses (2018, background): “Just as later work in the space of images showed black-box attacks are possible [101, 51]; we expect that our attacks can be extended to black-box attacks.”
Attacks on Machine Learning: Lurking Danger for Accountability (2019, background): “Another group of attacks that tamper with data fed into the ML model, and thus are applicable on a wide range of different ML algorithms, are adversarial examples [5, 6, 17, 34, 51, 59], evasion attacks [23, 78], and feature deletion attacks.”
VectorDefense: Vectorization as a Defense to Adversarial Examples (2018, methodology): “These adversarial perturbations can be as minute as one single pixel [37]; computed inexpensively with a single backward pass through the DNN [11] or via black-box optimization algorithms [7, 14, 29].”