Pretrained Transformers Improve Out-of-Distribution Robustness

By Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, R. Krishnan, Dawn Song. 2020

492 citations recorded by the source corpus.

Although pretrained Transformers such as BERT achieve high accuracy on in-distribution examples, do they generalize to new distributions? We systematically measure out-of-distribution (OOD) generalization for seven NLP datasets by constructing a new robustness benchmark with realistic distribution shifts. We measure the generalization of previous models including bag-of-words models, ConvNets, and LSTMs, and we show that pretrained Transformers’ performance declines are substantially smaller. Pretrained transformers are also more effective at detecting anomalous or OOD examples, while many previous models are frequently worse than chance. We examine which factors affect robustness, finding that larger models are not necessarily more robust, distillation can be harmful, and more diverse pretraining data can enhance robustness. Finally, we show where future work can improve OOD robustness.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Representation & Self-Supervised Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Representation & Self-Supervised Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Learning Transferable Visual Models From Natural Language Supervision (2021, background): “They measure performance on a set of 7 distribution shifts: ImageNetV2 (Recht et al., 2019), ImageNet Sketch (Wang et al., 2019), Youtube-BB and ImageNet-Vid (Shankar et al., 2019), ObjectNet (Barbu et al., 2019), ImageNet Adversarial (Hendrycks et al., 2019), and ImageNet Rendition (Hendrycks et al”
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, background): “Nevertheless, there has been mounting evidence in recent work suggesting that the performance of these models as measured by the benchmark datasets may be overestimated, non-generalizable and at least partially driven by exploiting existing spurious correlations in training datasets [21, 22, 40, 47,”
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “Hendrycks et al., 2020 [14] find that many recent language models are already robust to many forms of distribution shift, while others [37, 10] find that vision models are largely fragile and argue that data augmentation offers one solution.”
The Effect of Natural Distribution Shift on Question Answering Models (2020, methodology): “Hendrycks et al. (2020) evaluates robustness to distribution shift for question answering, among other tasks, by carefully splitting subsets of the ReCoRD dataset (Zhang et al., 2018).”
Robust Speech Recognition via Large-Scale Weak Supervision (2022, background): “An inspirational and foundational work in NLP exploring multi-task learning with a single model is Collobert et al. (2011).”
Progressive Prompts: Continual Learning for Language Models (2023, methodology): “We follow a similar transfer learning experimental protocol by Hendrycks et al. (2020) and use pairs of six tasks from similar domains: IMDb and SST2 (2-class sentiment classiﬁcation), Amazon and Yelp reviews (5-class sentiment classiﬁcation), MRPC and QQP (paraphrase detection).”
An Empirical Investigation of the Role of Pre-training in Lifelong Learning (2021, neutral): “, 2020), and out-of-distribution robustness (Hendrycks et al., 2020; Tu et al., 2020).”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “…et al., 2006; Daumé III, 2007; Axelrod et al., 2011), and has recently been addressed for neural LMs and transfer learning (Fried et al., 2019; Oren et al., 2019; Gururangan et al., 2020), where pretrained LMs lead to substantial improvements and increased robustness (Hendrycks et al., 2020).”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, background): “…Gardner et al., 2020), input037 perturbations (Wallace et al., 2019; Kaushik et al.,038 2020), and distribution shifts (Talmor and Berant,039 2019; Hendrycks et al., 2020).040 Dynamic adversarial data collection (DADC)041 holds promise as an approach to mitigate these042 training set problems.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “Others have studied proprietary datasets with realistic distribution shifts, such as the StreetView StoreFronts dataset (Hendrycks et al., 2020) or diabetic retinopathy datasets (D’Amour et al., 2020).”
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020, background): “Nevertheless, recent empirical results have suggested that self-supervised pre-training improves robust accuracy, while not using any task-speciﬁc knowledge nor incurring in-distribution accuracy drop (Hendrycks et al., 2019, 2020).”
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP (2021, background): “Hendrycks et al. (2020) studies the OOD generalization and OOD detection performance of BERT for a range of NLP tasks.”
Large Language Models Are Reasoning Teachers (2022, background): “However, given that ﬁne-tuning requires a very large dataset of task-speciﬁc labeled examples, and often does not generalize well to out-of-distribution settings, it is not always easily applicable (Liu et al., 2021; Hendrycks et al., 2020).”
How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks (2020, background): “…via representation learning with unlabeled out-of-distribution data from X \ D (Chen et al., 2020; Devlin et al., 2019; Hu et al., 2020; Peters et al., 2018), which might explain why pre-trained representations such as BERT can improve out-of-distribution robustness (Hendrycks et al., 2020).”
On the Robustness of Vision Transformers to Adversarial Examples (2021, methodology): “The transformer has been well studied from an adversarial perspective for NLP applications e.g., [18, 31, 17, 15].”
Improved OOD Generalization via Adversarial Training and Pre-training (2021, methodology): “We select training samples from two genres of transcribed text (Telephone and Face-to-Face) and the other of written text (Letters) as in (Hendrycks et al., 2020b), and report the classiﬁcation accuracy.”