BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance

By R. Thomas McCoy, Junghyun Min, Tal Linzen. 2019

163 citations recorded by the source corpus.

If the same neural network architecture is trained multiple times on the same dataset, will it make similar linguistic generalizations across runs? To study this question, we fine-tuned 100 instances of BERT on the Multi-genre Natural Language Inference (MNLI) dataset and evaluated them on the HANS dataset, which evaluates syntactic generalization in natural language inference. On the MNLI development set, the behavior of all instances was remarkably consistent, with accuracy ranging between 83.6% and 84.8%. In stark contrast, the same models varied widely in their generalization performance. For example, on the simple case of subject-object swap (e.g., determining that “the doctor visited the lawyer” does not entail “the lawyer visited the doctor”), accuracy ranged from 0.0% to 66.2%. Such variation is likely due to the presence of many local minima in the loss surface that are equally attractive to a low-bias learner such as a neural network; decreasing the variability may therefore require models with stronger inductive biases.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, methodology): “…systems: when tested on cases sampled from a distribution that differs from the one they were trained on, their behavior is unpredictable and inconsistent with that of humans (Jia and Liang, 2017; McCoy et al., 2019b), and they require extensive instruction on each new task (Yogatama et al., 2019).”
On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “[36] R Thomas McCoy, Junghyun Min, and Tal Linzen.”
How many data points is a prompt worth? (2021, background): “2 In line with previous observations (McCoy et al., 2019; Dodge et al., 2020; Lee et al., 2020), head-based fine-tuning performance varies considerably.”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, methodology): “We connect our results to previous work that reported instability in performance on stress tests designed to diagnose shortcut learning in Natural Language Inference tasks (McCoy et al., 2019b; Naik et al., 2018).”
Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, background): “Furthermore, while one might hope for generalizability after fine-tuning on a sufficient number of reasoning and symbol manipulation tasks with a common ‘language’ or protocol for representing action and state information, this may or may not be observed[43].”
Adversarial Filters of Dataset Biases (2020, methodology): “In particular, AFL ITE substantially outperforms challenging examples on the HANS benchmark, which targets models purely relying on lexical and syntactic cues.”
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020, methodology): “, 2019), and testing the linguistic generalizations of BERT across runs (McCoy et al., 2019).”
When BERT Plays the Lottery, All Tickets Are Winning (2020, background): “the performance of fine-tuned BERT (McCoy et al., 2019a; Dodge et al., 2020).”
With Little Power Comes Great Responsibility (2020, methodology): “in evaluation (McCoy et al., 2019; Azer et al., 2020), including the need for careful hypothesis testing (Koehn, 2004; Berg-Kirkpatrick et al.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “In natural language processing (NLP), a current focus is on challenge datasets that are carefully crafted to test particular aspects of models, e.g., HANS (McCoy et al., 2019), PAWS (Zhang et al., 2019), and CheckList (Ribeiro et al., 2020).”
The MultiBERTs: BERT Reproductions for Robustness Analysis (2021, background): “That is, evaluating our MNLI models on the anti-sterotypical examples from HANS (McCoy et al., 2019), we see agree- 4We used https://storage.googleapis. com/bert_models/2020_02_20/uncased_ L-12_H-768_A-12.zip, as linked from https: //github.com/google-research/bert ment drop from 88% to 82% when com”
Is neural language acquisition similar to natural? A chronological probing study (2022, background): “It is believed (McCoy et al., 2020), that rediscov-ering these structures would help models to get closer to human performance on a variety of tasks.”
UnNatural Language Inference (2020, background): “This is especially true for syntactic phenomena (Marvin and Linzen, 2018; Hu et al., 2020b; Gauthier et al., 2020; McCoy et al., 2020; Warstadt et al., 2020), where transformers are, for some phenomena and”
Learning from others' mistakes: Avoiding dataset biases without modeling them (2020, methodology): “In contrast to our experiments on MNLI/HANS, multi-loss training thus leads here to better performance on out-of-distribution as well.”
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning (2021, background): “McCoy et al. (2020) ﬁnd that even when the model’s architecture is set, the generalization ability of the model is still inﬂuenced largely by the random luck, the random initialized weights and other things.”
mGPT: Few-Shot Learners Go Multilingual (2022, background): “LMs can learn spurious correlations from finetuning data (Naik et al., 2018; Niven and Kao, 2019) and demonstrate inconsistent generalization, catastrophic forgetting, or brittleness to finetuning data order (McCoy et al., 2020; Dodge et al., 2020).”