Achieving Robustness in the Wild via Adversarial Mixing With Disentangled Representations

By Sven Gowal, Chongli Qin, Po-Sen Huang, taylan. cemgil, Krishnamurthy (Dj) Dvijotham, Timothy A. Mann, Pushmeet Kohli. 2019

60 citations recorded by the source corpus.

Recent research has made the surprising finding that state-of-the-art deep learning models sometimes fail to generalize to small variations of the input. Adversarial training has been shown to be an effective approach to overcome this problem. However, its application has been limited to enforcing invariance to analytically defined transformations like lp-norm bounded perturbations. Such perturbations do not necessarily cover plausible real-world variations that preserve the semantics of the input (such as a change in lighting conditions). In this paper, we propose a novel approach to express and formalize robustness to these kinds of real-world transformations of the input. The two key ideas underlying our formulation are (1) leveraging disentangled representations of the input to define different factors of variations, and (2) generating new input images by adversarially composing the representations of different images. We use a StyleGAN model to demonstrate the efficacy of this framework. Specifically, we leverage the disentangled latent representations computed by a StyleGAN model to generate perturbations of an image that are similar to real-world variations (like adding make-up, or changing the skin-tone of a person) and train models to be invariant to these perturbations. Extensive experiments show that our method improves generalization and reduces the effect of spurious correlations (reducing the error rate of a "smile" detector by 21% for example).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

On the Transfer of Disentangled Representations in Realistic Settings (2020, neutral): “However, these representations are trained on Gowal et al. (2020).”
Improving the accuracy of medical diagnosis with causal machine learning (2020, background): “The question of how to incorporate causal and counter-factual reasoning into other machine learning methods beyond structural causal models, for example in Deep Learning for image classi ﬁ cation 82,83 and deep generative models 84 – 86 , is an active area research.”
Fixing Data Augmentation to Improve Adversarial Robustness (2021, methodology): “Finally, we report the robust test accuracy against a mixture of AUTOATTACK (Croce & Hein, 2020b) and MULTITARGETED (Gowal et al., 2019b), which is denoted by AA+MT.”
Adversarial Robustness with Semi-Infinite Constrained Learning (2021, result): “In particular, our results apply to arbitrary perturbation sets, such as those used in [62–66].”
Ensembling with Deep Generative Views (2021, methodology): “[49] demonstrate projecting adversarial images through a generator on MNIST and Fashion-MNIST, while other works [51, 25, 19] train models using the generator to improve robustness.”
Improving Robustness using Generated Data (2021, background): “When optimized and used during training, these novel views reduce the impact of spurious correlations and improve accuracy [28, 73].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[333] improved adversarial training by varying the Style-GAN-based [334] disentangled representations of original images.”