On the Suitability of Lp-Norms for Creating and Preventing Adversarial Examples

By Mahmood Sharif, Lujo Bauer, Michael K. Reiter. 2018

146 citations recorded by the source corpus.

Much research has been devoted to better understanding adversarial examples, which are specially crafted inputs to machine-learning models that are perceptually similar to benign inputs, but are classified differently (i.e., misclassified). Both algorithms that create adversarial examples and strategies for defending against adversarial examples typically use Lp-norms to measure the perceptual similarity between an adversarial input and its benign original. Prior work has already shown, however, that two images need not be close to each other as measured by an Lp-norm to be perceptually similar. In this work, we show that nearness according to an Lp-norm is not just unnecessary for perceptual similarity, but is also insufficient. Specifically, focusing on datasets (CIFAR10 and MNIST), Lp-norms, and thresholds used in prior work, we show through online user studies that "adversarial examples" that are closer to their benign counterparts than required by commonly used Lp-norm thresholds can nevertheless be perceptually distinct to humans from the corresponding benign examples. Namely, the perceptual distance between two images that are "near" each other according to an Lp-norm can be high enough that participants frequently classify the two images as representing different objects or digits. Combined with prior work, we thus demonstrate that nearness of inputs as measured by Lp-norms is neither necessary nor sufficient for perceptual similarity, which has implications for both creating and defending against adversarial examples. We propose and discuss alternative similarity metrics to stimulate future research in the area.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Training for High-Stakes Reliability (2022, background): “adversarial training work consists of studying Lp ball perturbations [22, 28, 29, 30, 31].”
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “The work of Sharif et al. (2018) also shows that “small” `1 noise (of magnitude 25/255) can reliably fool human labelers on CIFAR-10.”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “Sharif et al. [45] empirically show that (a) two images that are perceptually similar to humans can have a large l p distance, and (b) two images with a small l p distance can have different semantics.”
Disentangling Adversarial Robustness and Generalization (2018, background): “Generalization: Research also includes independent benchmarks of attacks and defenses [13, 15, 5, 6, 85], their properties [57, 84], as well as theoretical questions [34, 42, 23, 99, 27, 88, 102, 104].”
PRADA: Protecting Against DNN Model Stealing Attacks (2018, methodology): “Alternatively, any L norm or the structural similarity metric [45] could be used for image input.”
Revisiting Adversarial Risk (2018, background): “the margin condition?” Sharif et al. [2018] try to estimate the margin in MNIST, CIFAR10 datasets via user studies.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “Whereas the majority of mainstream attacks induce perturbation imperceptibility in adversarial images by restricting the `p-norm of perturbations, it is sometimes argued that the perturbation norm is not a good indicator of the perceptual difference between the two images [121].”