Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors

By Andrew Ilyas, Logan Engstrom, Aleksander Madry. 2018

430 citations recorded by the source corpus.

We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available. We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and we demonstrate that the current state-of-the-art methods are optimal in a natural sense. Despite this optimality, we show how to improve black-box attacks by bringing a new element into the problem: gradient priors. We give a bandit optimization-based algorithm that allows us to seamlessly integrate any such priors, and we explicitly identify and incorporate two examples. The resulting methods use two to four times fewer queries and fail two to five times less often than the current state-of-the-art.

Subfields: Artificial Intelligence, Social Psychology, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Social Psychology

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Adversarial Example Games (2020, methodology): “…input-output pairs from the target model, —i.e. logits, ﬁnal prediction, or even a ﬁnite resource in the number of staged queries [15, 23, 38–40 , In this paper, we consider the challenging setting of no n-interactive black Box ( NoBox ) attacks, intending to generate successful attacks…”
On Evaluating Adversarial Robustness (2019, background): “• NES (Ilyas et al., 2018a) is effective at generating adversarial examples with a limited number of queries, and so generates adversarial examples of higher distortion.”
Disentangling Adversarial Robustness and Generalization (2018, background): “), to black-box attacks [17, 10, 96, 38, 80, 67], with limited access to model queries.”
Square Attack: a query-efficient black-box adversarial attack via random search (2019, methodology): “We compare our attack to Bandits [31] and SimBA [28] for = 5, while we do not consider SignHunter [2] since it is not as competitive as for the l∞-norm, and in particular worse than Bandits on ImageNet (see Fig.”
Security and Privacy Issues in Deep Learning (2018, methodology): “ to unseen attacks by identifying blind spots that should be considered and addressed. In this sense, security issues should be constantly researched and developed considering different perspectives [Ilyas et al. 2019; Tsipras et al. 2019] on adversarial attacks. For the second part of Section 3, we”
Boosting Decision-Based Black-Box Adversarial Attacks with Random Sign Flip (2020, methodology): “The method can be further improved by exploiting two kinds of gradient priors [28].”
Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks (2018, background): “data used are 5,000 random images from the validation set of the ILSVRC2012 ImageNet classification task [63].”
A Survey of Black-Box Adversarial Attacks on Computer Vision Models (2019, methodology): “[23] Bandits &Priors Jul 2019 Alzantot et al.”
Simple Black-box Adversarial Attacks (2019, background): “To date, the average number of queries performed by the best known black-box attacks remains high despite a large amount of recent work in this area (Chen et al., 2017; Bren-del et al., 2017; Cheng et al., 2018; Guo et al., 2018; Tu et al., 2018; Ilyas et al., 2018a).”
HopSkipJumpAttack: A Query-Efficient Decision-Based Attack (2019, methodology): “Zeroth-order algorithms have been applied to the generation of adversarial examples under the score-based threat model [8–10].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, positive): “[88] revisited the zeroth-order optimization (zoo) and proposed a query-based attack using bandit optimization that exploits prior information about the target model gradient.”
Query-efficient Meta Attack to Deep Neural Networks (2019, methodology): “Another strategy is to reformulate the loss function (Ilyas et al., 2018a;b).”
Relating Adversarially Robust Generalization to Flat Minima (2021, background): “Adversarial Examples and Defenses: Adversarial examples, first reported in [122], can be generated using a wide-range of white-box attacks [122, 40, 67, 94, 87, 80, 14, 32, 78], with full access to the network, or black-box attacks [18, 11, 120, 54, 107, 88], with limited access to model queries.”
Adversarially Robust Neural Architectures (2020, neutral): “Besides transfer-based closed-box attacks, some query-based attacks were introduced where adversaries can only query the outputs of the models [28], [34].”
Black-box Prompt Learning for Pre-trained Language Models (2022, background): “Ilyas et al. (2019b) find that neural net- works rely on some non-robust features to achieve the highest possible accuracy.”
Improving Black-box Adversarial Attacks with a Transfer-based Prior (2019, methodology): “We further incorporate the data-dependent prior [19] into our algorithm.”
Black-Box Adversarial Attack with Transferable Model-based Embedding (2019, methodology): “As in (Ilyas et al., 2018a; Wierstra et al., 2014), we can employ NES to approximate the gradient of a properly deﬁned surrogate loss in order to ﬁnd an adversarial example.”