Analysis of universal adversarial perturbations

By Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, Stefano Soatto. 2017

122 citations recorded by the source corpus.

Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we propose the first quantitative analysis of the robustness of classifiers to universal perturbations, and draw a formal link between the robustness to universal perturbations, and the geometry of the decision boundary. Specifically, we establish theoretical bounds on the robustness of classifiers under two decision boundary models (flat and curved models). We show in particular that the robustness of deep networks to universal perturbations is driven by a key property of their curvature: there exists shared directions along which the decision boundary of deep networks is systematically positively curved. Under such conditions, we prove the existence of small universal perturbations. Our analysis further provides a novel geometric method for computing universal perturbations, in addition to explaining their properties.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Computer Vision and Pattern Recognition, Neurology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Computer Vision and Pattern Recognition

In Neurology

In Safety, Robustness & Interpretability

Citations in the corpus

ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, background): “Further analysis and justification of a universal perturbation is given in [30].”
Efficient Defenses Against Adversarial Attacks (2017, background): “In [28, 30], the authors extend DeepFool in order to craft a universal perturbation to be applied indifferently to any instance: a fixed distortion is computed from a set of inputs, allowing to maximize the predictive error of the model on that sample.”
One Pixel Attack for Fooling Deep Neural Networks (2017, background): “Thus, usual adversarial samples are constructed by perturbating all pixels with an overall constraint on the strength of accumulated modification[14, 8] while the few-pixel attack considered in this paper is the opposite which specifically focus on few pixels but does not limit the strength of modif”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “In [70], they built further on their theory and showed the existence of common directions (shared across datapoints) along which the decision boundary”
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization (2018, background): “In [25] the authors show the link between a network’s robustness to adversarial perturbations and the geometry of its decision boundaries.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “The authors theoretically demonstrate the existence of common directions for multiple data points along which a classifier’s decision boundaries can be highly curved [319],”
Adversarial Examples on Object Recognition (2020, background): “lesareaconsequenceofnon-robustfeatures, which arederived frompatternsin thedata thatcanbeeasy topredictbycomputers, butnot understood by humans. Relation to decisionboundaries. Moosavi-Dezfooli et al.[121]showedthatit ispossibletogenerate universal perturbations – which can be applied to any input. ”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, background): “close to decision boundary of a natural trained model, which may be because the decision boundary is too flat[92], too curved[93], or inflexible[94].”
Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation (2018, background): “dataset. 2.3 Theoretical Aspects Theoretical aspects concern mainly the understanding and the provability of DNNs [145,194,195,219], but also their properties in presence of adversarial perturbations [34,137,138,139,140,186,231],andtheirrobustnessinpresenceofnoisylabels[189]. For this, the principle”
Universal Adversarial Training (2018, methodology): “…universal perturbation Given a set of training samples X = { x i , i = 1 , . . . , N } and a network f ( w, · ) with frozen parameter w that maps images onto labels, Moosavi-Dezfooli et al. (2017b) propose to ﬁnd universal perturbations δ that satisfy, Prob ( X, δ ) represents the “fooling…”
Generative Adversarial Perturbations (2017, background): “[36] proposes a quantitative analysis of the robustness of classifiers to universal perturbations based on the geometric properties of decision boundaries.”
A Geometric Perspective on the Transferability of Adversarial Directions (2018, background): “As demonstrated by Moosavi-Dezfooli et al. (2017a), Narodytska and Kasiviswanathan (2016), and Papernot et al. (2016), adversarial examples often transfer to other classiﬁers.”
A Survey On Universal Adversarial Attack (2021, background): “(1) A popular choice is to set p =∞, and to set the value of to 10/255, assuming images to be in the range [0, 1] [MoosaviDezfooli et al., 2017a; Poursaeed et al., 2018; Mopuri et al., 2018a].”
With Friends Like These, Who Needs Adversaries? (2018, methodology): “They illustrate in [5] why a hypothetical network which possessed this property would theoretically be predicted to be vulnerable to universal adversaries, and note that the analysis suggests a direct construction method for such adversaries as an alternative to the original randomised iterative app”