On the Sensitivity of Adversarial Robustness to Input Data Distributions

By Gavin Weiguang Ding, Kry Yik-Chau Lui, Xiaomeng Jin, Luyu Wang, Ruitong Huang. 2018

67 citations recorded by the source corpus.

Neural networks are vulnerable to small adversarial perturbations. Existing literature largely focused on understanding and mitigating the vulnerability of learned models. In this paper, we demonstrate an intriguing phenomenon about the most popular robust training method in the literature, adversarial training: Adversarial robustness, unlike clean accuracy, is sensitive to the input data distribution. Even a semantics-preserving transformations on the input data distribution can cause a significantly different robustness for the adversarial trained model that is both trained and evaluated on the new distribution. Our discovery of such sensitivity on data distribution is based on a study which disentangles the behaviors of clean accuracy and robust accuracy of the Bayes classifier. Empirical investigations further confirm our finding. We construct semantically-identical variants for MNIST and CIFAR10 respectively, and show that standardly trained models achieve comparable clean accuracies on them, but adversarially trained models achieve significantly different robustness accuracies. This counter-intuitive phenomenon indicates that input data distribution alone can affect the adversarial robustness of trained neural networks, not necessarily the tasks themselves. Lastly, we discuss the practical implications on evaluating adversarial robustness, and make initial attempts to understand this complex phenomenon.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial T-Shirt! Evading Person Detectors in a Physical World (2019, background): “ations are studied via parametric models. Other transformations including perspective transformation, brightness adjustment, resampling (or image resizing), smoothing and saturation are considered in [29, 9].All the existing transformations are included in our library of physical transformations. Ho”
advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch (2019, background): “…methodologies (Weng et al., 2018; Katz et al., 2017; Athalye et al., 2018); and understanding the vulnerability phenomena (Fawzi et al., 2017; Shafahi et al., 2019; Ding et al., 2019). advertorch aims to provide researchers the tools for conducting research in all the above mentioned directions.”
MMA Training: Direct Input Space Margin Maximization through Adversarial Training (2018, background): “Although previous literature (Ding et al., 2019a; Zhang et al., 2019c) also discusses related topics on the difference between MNIST and CIFAR10 w.r.t. adversarial robustness, they do not directly explain the observed phenomena here.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “It is also claimed that adversarial training is sensitive to the training data distribution in [327].”
Improving Adversarial Robustness Requires Revisiting Misclassified Examples (2020, background): “However, the condition cannot be directly optimized if we conduct a hard decision during the training process (Ding et al. (2019) uses hard decision and does not optimize the condition).”