Intriguing properties of adversarial training

By Cihang Xie, Alan L. Yuille. 2019

65 citations recorded by the source corpus.

Adversarial training is one of the main defenses against adversarial attacks. In this paper, we provide the first rigorous study on diagnosing elements of adversarial training, which reveals two intriguing properties. First, we study the role of normalization. Batch normalization (BN) is a crucial element for achieving state-of-the-art performance on many vision tasks, but we show it may prevent networks from obtaining strong robustness in adversarial training. One unexpected observation is that, for models trained with BN, simply removing clean images from training data largely boosts adversarial robustness, i.e., 18.3%. We relate this phenomenon to the hypothesis that clean images and adversarial images are drawn from two different domains. This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging. Guided by this two-domain hypothesis, we show disentangling the mixture distribution for normalization, i.e., applying separate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness. Additionally, we find that enforcing BNs to behave consistently at training and testing can further enhance robustness. Second, we study the role of network capacity. We find our so-called "deep" networks are still shallow for the task of adversarial learning. Unlike traditional classification tasks where accuracy is only marginally improved by adding more layers to "deep" networks (e.g., ResNet-152), adversarial training exhibits a much stronger demand on deeper networks to achieve higher adversarial robustness. This robustness improvement can be observed substantially and consistently even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “• Larger Models: increasing model size improves robustness to distribution shift [12, 35].”
Convergence of Adversarial Training in Overparametrized Neural Networks (2019, unknown): “Apart from this paper, there are other works supporting the need for capacity including the perspective of network width [31], depth [49] and computational complexity [35].”
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (2019, background): “To mitigate this, large efforts have been devoted to adversarial defense [7, 34, 47, 72], via adversarial training [42, 48, 69, 76, 68, 77, 74], randomized transformation [14, 26, 45, 75] etc.”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, methodology): “Larger models provide improved robustness (Madry et al., 2017; Xie & Yuille, 2019b; Uesato et al., 2019) and, for identical parameter count, deeper models can perform better.”
Improving robustness against common corruptions by covariate shift adaptation (2020, background): “Robustness scores obtained by adversarial training can be improved when separate BN or GroupNorm layers are used for clean and adversarial images [53].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[279] C. Xie, M. Tan, B. Gong, J. Wang, A. L. Yuille, and Q. V. Le, ‘‘Adversarial examples improve image recognition,’’ in Proc.”
ResNet or DenseNet? Introducing Dense Shortcuts to ResNet (2020, background): “Additionally, the effect of batch normalization on adversarial robustness has been performed in [52, 2].”
When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks (2019, background): “A few empirical attempts on robustness of existing network architectures have been made [30, 36], but no convincing guidelines or conclusions have yet been achieved.”