Simple Black-box Adversarial Attacks

By Chuan Guo, Jacob V. Gardner, Yurong You, Andrew Gordon Wilson, Kilian Q. Weinberger. 2019

714 citations recorded by the source corpus.

We propose an intriguingly simple method for the construction of adversarial images in the black-box setting. In constrast to the white-box scenario, constructing black-box adversarial images has the additional constraint on query budget, and efficient attacks remain an open problem to date. With only the mild assumption of continuous-valued confidence scores, our highly query-efficient algorithm utilizes the following simple iterative principle: we randomly sample a vector from a predefined orthonormal basis and either add or subtract it to the target image. Despite its simplicity, the proposed method can be used for both untargeted and targeted attacks -- resulting in previously unprecedented query efficiency in both settings. We demonstrate the efficacy and efficiency of our algorithm on several real world settings including the Google Cloud Vision API. We argue that our proposed algorithm should serve as a strong baseline for future black-box attacks, in particular because it is extremely fast and its implementation requires less than 20 lines of PyTorch code.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Example Games (2020, background): “This setting is more permissive than the semi-whtiebox and the blackbox setting, the latter of which the attacker has only access to the prediction [54, 55] or sometimes the predicted conﬁdence [33].”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “Adversarial examples can be generated using access to model parameters, also known as whitebox attacks, or without such access, with blackbox attacks (Papernot et al., 2016a, 2017; Narodytska and Kasiviswanathan, 2017; Liu et al., 2017).”
A Fourier Perspective on Model Robustness in Computer Vision (2019, methodology): “In contrast, methods for generating small adversarial perturbations require 1000’s of queries [14].”
Square Attack: a query-efficient black-box adversarial attack via random search (2019, background): “We compare our attack to Bandits [31] and SimBA [28] for = 5, while we do not consider SignHunter [2] since it is not as competitive as for the l∞-norm, and in particular worse than Bandits on ImageNet (see Fig.”
Explaining in Style: Training a GAN to explain a classifier in StyleSpace (2021, background): “Adversarial examples [9, 12] , which are slight modifications to the input x that change the classification to the wrong class, are also not effective counterfactual explanations, as the changes are usually not interpretable by humans.”
Towards Frequency-Based Explanation for Robust CNN (2020, background): “For SimBA, we used step size of 0.2 and we clip the distortion in the l∞ ball with the maximum allowed perturbation 0.2.”
Meta-Learning with Neural Tangent Kernels (2021, positive): “For Black-box attack, we adopt the strong query efﬁcient attack method (Guo et al., 2019).”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “4 plot the expected calibration error (ECE) without and with temperature rescaling [54] to minimize the ECE (which is a simple but effective post-hoc calibration method, see Appendix F for details) together with the optimal temperature for a large set of `∞ models.”
Black-Box Adversarial Attack with Transferable Model-based Embedding (2019, background): “Although it also produces structural perturbations, these perturbations are usually not suitable for attacking regular networks and sometimes its performance is even worse than directly applying NES to the images (Cheng et al., 2019; Guo et al., 2019).”
Boosting Decision-Based Black-Box Adversarial Attacks with Random Sign Flip (2020, background): “Instead of gradient approximation, some attacks work in a gradient-free manner [1, 2, 22, 36].”
A Survey of Black-Box Adversarial Attacks on Computer Vision Models (2019, background): “• SimBA: The study shown in [25] proposes a straightforward yet efficient approach to generate adversarial examples in a black-box setting.”
TrojanNet: Embedding Hidden Trojan Horse Models in Neural Networks (2020, methodology): “This attack has been successfully launched against real world systems such as Google Voice (Cisse et al., 2017), Clarifai (Liu et al., 2016) and Google Cloud Vision (Ilyas et al., 2018; Guo et al., 2018; 2019).”
Black-box Prompt Learning for Pre-trained Language Models (2022, methodology): “One of the applications of BLACK-BOX optimization is the score-based BLACK-BOX adversarial attack (Ilyas et al., 2018a,b; Huang and Zhang, 2019; Andriushchenko et al., 2020; Cheng et al., 2019; Guo et al., 2019), where the models are also invisible to the attacker.”