Universal Adversarial Perturbations

By Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard. 2016

2,828 citations recorded by the source corpus.

Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that causes natural images to be misclassified with high probability. We propose a systematic algorithm for computing universal perturbations, and show that state-of-the-art deep neural networks are highly vulnerable to such perturbations, albeit being quasi-imperceptible to the human eye. We further empirically analyze these universal perturbations and show, in particular, that they generalize very well across neural networks. The surprising existence of universal perturbations reveals important geometric correlations among the high-dimensional decision boundary of classifiers. It further outlines potential security breaches with the existence of single directions in the input space that adversaries can possibly exploit to break a classifier on most natural images.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

A Study and Comparison of Human and Deep Learning Recognition Performance under Visual Distortions (2017, background): “The distortions for these models can be seen as a type of universal adversarial perturbation [20] that forces the model to predict a particular class for any input.”
Adversarial Examples for Evaluating Reading Comprehension Systems (2017, background): “In contrast with prior work in computer vision (Papernot et al., 2017; Narodytska and Kasiviswanathan, 2016; Moosavi-Dezfooli et al., 2017), ADDONESENT does not require any access to the model or to any training data: it generates adversarial examples based solely on the intuition that existing mode”
Universal Adversarial Triggers for Attacking and Analyzing NLP (2019, background): “Triggers are a new form of universal adversarial perturbation (Moosavi-Dezfooli et al., 2017) adapted to discrete textual inputs.”
Semantically Equivalent Adversarial Rules for Debugging NLP models (2018, background): “Adversaries are usually specific to predictions, and even universal adversarial perturbations (Moosavi-Dezfooli et al., 2017) are not natural, semantically meaningful to humans, or actionable.”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “been shown for changing a network’s classification of any image to any target class by making imperceptible alterations to the pixels [3], [4], [5].”
Deep Variational Information Bottleneck (2017, background): “Most attacks fall into three broad categories: optimization-based attacks (Szegedy et al., 2014; Carlini & Wagner, 2016; Moosavi-Dezfooli et al., 2016; Papernot et al., 2015; Robinson & Graham, 2015; Sabour et al., 2016), which directly run an optimizer such as L-BFGS or ADAM (Kingma & Ba, 2015) on…”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, methodology): “mage that is imperceptible to humans. Numerous works have built on this approach; some notable works include the Fast Gradient Sign Method [18], DeepFool [35], and Universal Adversarial Perturbations [34]. This area of research can be categorized as a cat-and-mouse game in recent years, where defens”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “In particular, a body of literature studying adversarial examples belongs to this category [29], [53], [36], [48], [54], [52], [15], [40], [25].”
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain (2017, background): “Follow-on work improved the speed at which adversarial examples could be created [33], demonstrated that adversarial examples could be found even if only black-box access to the target model was available [34], and even discovered universal adversarial perturbations [35] that could cause different i”
ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, background): “More interestingly, the authors in [29] have shown that a carefully crafted universal perturbation to a set of natural images can lead to misclassification of all considered images with high probability, suggesting the possibility of attack transferability from one image to another.”
Backdooring Convolutional Neural Networks via Targeted Weight Perturbations (2018, background): “These are images that have been perturbed in some way that cause misclassifications when given as inputs to these networks [7, 21, 23, 25, 29, 22].”
Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations (2018, background): “Beyond data inference, MLmodels have been shown to be subject to a variety of other attacks, such as, adversarial attacks on object detectors [27] or classifiers [13, 29, 30, 32], malicious algorithms which discreetly memorize the training data samples [39], and malicious classifiers which mislead t”
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization (2017, background): “Besides the extension and evaluation of this poisoning attack strategy to different deep learning architectures and nonparametric models, further research avenues include: the investigation of the existence of universal perturbations (not dependent on the initial attack point) for poisoning samples ”
On Detecting Adversarial Perturbations (2017, methodology): “Actually, Moosavi-Dezfooli et al. (2016a) show that universal perturbations are not unique and that there even exist many different universal perturbations which have little in common.”
Delving into Transferable Adversarial Examples and Black-box Attacks (2016, background): “In a concurrent and independent work, Moosavi-Dezfooli et al. (2016) showed the existence of a universal perturbation for each model, which can transfer across different images.”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “Since the seminal work of [42] exhibiting this intriguing phenomenon in the context of deep learning, a wealth of results have been published on designing attacks [18, 34, 32, 23, 6, 31] and defenses [18, 35, 20, 29, 39, 27]), or on trying to understand the very nature of this phenomenon [17, 40, 15”
Spatially Transformed Adversarial Examples (2018, background): “…et al., 2015; Carlini & Wagner, 2017a; Moosavi-Dezfooli et al., 2015; Papernot et al., 2016b; Biggio et al., 2013; Kurakin et al., 2016); while having zero knowledge about them in the black-box setting (Papernot et al., 2016a; Liu et al., 2017; Moosavi-Dezfooli et al., 2016; Mopuri et al., 2017).”
Robust Physical-World Attacks on Deep Learning Models (2017, background): “Universal Perturbations is another untargeted attack for generating adversarial examples, but rather then generating an adversarial input, the attack generates a perturbation vector [19].”