Adversarial Robustness as a Prior for Learned Representations

By Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, Aleksander Madry. 2019

240 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “Interestingly, Engstrom et al. (2019a) show an opposite effect for models’ internal representations.”
Robustness properties of Facebook's ResNeXt WSL models (2019, methodology): “Engstrom et al. (2019) recently showed that models trained with robust optimization learn fundamentally different features from models trained in the standard way (through minimization of training loss).”
Partial success in closing the gap between human and machine vision (2021, background): “The resulting models were found to transfer better, have meaningful gradients [65], and enable interpolating between two input images [66]: “robust optimization can actually be viewed as inducing a human prior over the features that models are able to learn” [67, p. 10].”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “However, despite the simplicity of the `p-perturbation 95 model, it has numerous interesting applications that go beyond security considerations [126, 105] 96 and span transfer learning [106, 130], interpretability [128, 65, 36], generalization [142, 156, 9], 97 robustness to unseen perturbations [6”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “State-of-the-art defenses on ImageNet use adversarial training [87, 41, 66, 100] or randomized smoothing [17], and images generated by optimizing class conﬁdence under these models are more perceptually recognizable to humans [80, 30, 93, 51].”
A Self-supervised Approach for Adversarial Robustness (2020, methodology): “VGG16-NT and ResNet50-NT are trained on ImageNet while ResNet50-AT [6] is adversarially trained on a subset of ImageNet.”
Datamodels: Predicting Predictions from Training Data (2022, background): “…can serve as an effective proxy for visual similarity (Barz & Denzler, 2020; Zhang et al., 2018), and also enable a suite of applications such as clustering (Guérin et al., 2017) and feature visualization (Olah et al., 2017; Engstrom et al., 2019; Azizpour et al., 2015; Ben-David et al., 2007).”
Media Forensics and DeepFakes: An Overview (2020, background): “These include content generation [22], [23], image/video synthesis from semantic labels [24], [25] or sketches [26] or text [27], changes of style and attributes [22], [28], [29], domain translation [30], up to expression transfer [25], face swapping typical of deepfakes [31] and talking-head video ”
On Adaptive Attacks to Adversarial Example Defenses (2020, methodology): “That is, an unbounded attack that maximizes the score of a detector results in semantically meaningful images from that class—mirroring similar results for adversarially trained classiﬁers (Santurkar et al., 2019).”
Out-of-Distribution Generalization via Risk Extrapolation (REx) (2020, background): “…has become a major research topic, with many different threads of research including Bayesian deep learning (Neal, 1996; Gal, 2016), adversarial (Engstrom et al., 2019; Jacobsen et al., 2018) and non-adversarial (Hendrycks & Dietterich, 2019; Yin et al., 2019) robustness, causality (Arjovsky et…”
Does language help generalization in vision models? (2021, methodology): “The different model variants (AR-L2,AR-LI4,AR-LI8) correspond to distinct adversarial attacks (refer to [3] for more details).”