The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

By Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, R. Desai, Tyler Lixuan Zhu, Samyak Parajuli, Mike Guo and 3 more. 2020

2,442 citations recorded by the source corpus.

We introduce four new real-world distribution shift datasets consisting of changes in image style, image blurriness, geographic location, camera operation, and more. With our new datasets, we take stock of previously proposed methods for improving out-of-distribution robustness and put them to the test. We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work. We find improvements in artificial robustness benchmarks can transfer to real-world distribution shifts, contrary to claims in prior work. Motivated by our observation that data augmentations can help with real-world distribution shifts, we also introduce a new data augmentation method which advances the state-of-the-art and outperforms models pre-trained with 1000× more labeled data. Overall we find that some methods consistently help with distribution shifts in texture and local image statistics, but these methods do not help with some other distribution shifts like geographic changes. Our results show that future research must study multiple distribution shifts simultaneously, as we demonstrate that no evaluated method consistently improves robustness.

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “They measure performance on a set of 7 distribution shifts: ImageNetV2 (Recht et al., 2019), ImageNet Sketch (Wang et al., 2019), Youtube-BB and ImageNet-Vid (Shankar et al., 2019), ObjectNet (Barbu et al., 2019), ImageNet Adversarial (Hendrycks et al., 2019), and ImageNet Rendition (Hendrycks et al”
On Robustness and Transferability of Convolutional Neural Networks (2020, background): “In [26] the authors list several of the hypotheses that appear in the literature, and collect new datasets that provide (both positive and negative) evidence for their soundness.”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, background): “…because scaling up data and model size has become a popular strategy for improving performance across a wide range of robustness stress tests, aligning closely with how much this scaling improves performance on iid evaluations (Djolonga et al., 2020; Taori et al., 2020; Hendrycks et al., 2020).”
Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “Existing work typically argues that an intervention improves robustness by showing that the accuracy on a robustness test set has improved (e.g., see [34, 40, 63, 102, 115]).”
Natural Adversarial Examples (2019, background): “Our examples demonstrate that it is possible to reliably fool many models with clean natural images, while previous attempts at exposing and measuring model fragility rely on synthetic distribution corruptions [18, 26], artistic renditions [24], and adversarial distortions.”
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022, methodology): “Accuracy is averaged over ImageNet and the ﬁve distribution shifts described in Section 3.1. the ﬁve natural distribution shifts: ImageNetV2 (Recht et al., 2019), ImageNet-R (Hendrycks et al., 2021a), ImageNet-Sketch (Wang et al., 2019), ObjectNet (Barbu et al., 2019), and ImageNet-A (Hendrycks et a”
Combined Scaling for Zero-shot Transfer Learning (2021, methodology): “2 on 5 robustness benchmarks derived from ImageNet: ImageNet-A [31], ImageNet-R [30], ImageNet-V2 [65], ImageNet-Sketch [89], and ObjectNet [4].”
LiT: Zero-Shot Transfer with Locked-image text Tuning (2021, methodology): “We evaluate robustness on ImageNetv2 [48], -R [26, 63], -A [27], -ReaL [2], and ObjectNet [1], following CLIP and ALIGN.”
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision (2021, background): “, 2009) and its variants including ImageNet-R(endition) (Hendrycks et al., 2020) (non-natural images such as art, cartoons, sketches), ImageNet-A(dversarial) (Hendrycks et al.”
Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement (2023, methodology): “We also evaluate ImageNet models on other distribution shift datasets, ImageNet-A [24], ImageNet-R [22], ImageNet-Sketch [61], ObjectNet [4], and ImageNet-C [23].”
LAION-5B: An open large-scale dataset for training next generation image-text models (2022, methodology): “We follow [94] and evaluate robustness performance on ImageNet distribution shift datasets [3, 23, 25, 61, 82].”
Unsolved Problems in ML Safety (2021, background): “In addition to existing robustness benchmarks [78, 102, 75], researchers could create more environments and benchmarks to stress-test systems, ﬁnd their breaking points, and determine whether they will function appropriately in potential future scenarios.”
SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model (2023, neutral): “ImageNet-R dataset [15] contains 200-class images, spliting into 24,000 and 6,000 images for training and testing (similar ratio for each class), respectively.”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “benchmark We also evaluate on a challenging dual-shift benchmark using the ImageNet-R dataset.”
A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning (2022, methodology): “Results Table 1, 4, and 2 report the performance of continual learning in incremental setting in Average Accuracy at the end of the learning sequence of Split-CIFAR100, Split-ImageNet-R, and the 5-dataset respectively.”
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, methodology): “Moreover, we introduce Split ImageNet-R, a new CL benchmark based on ImageNet-R [15] to the community.”
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions (2022, background): “This has resulted in the proliferation of new concepts and settings such as out-of-distribution generalization [15], domain generalization [3], multi-task learning [39], continual learning [23] and etc. Continual learning , which is the focus of this paper, concerns learning through a sequence of…”