Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets

By Nelson F. Liu, Roy Schwartz, Noah A. Smith. 2019

113 citations recorded by the source corpus.

Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks. While model performance on these challenge datasets is significantly lower compared to the original benchmark, it is unclear what particular weaknesses they reveal. For example, a challenge dataset may be difficult because it targets phenomena that current models cannot capture, or because it simply exploits blind spots in a model’s specific training set. We introduce inoculation by fine-tuning, a new analysis method for studying challenge datasets by exposing models (the metaphorical patient) to a small amount of data from the challenge dataset (a metaphorical pathogen) and assessing how well they can adapt. We apply our method to analyze the NLI “stress tests” (Naik et al., 2018) and the Adversarial SQuAD dataset (Jia and Liang, 2017). We show that after slight exposure, some of these datasets are no longer challenging, while others remain difficult. Our results indicate that failures on challenge datasets may lead to very different conclusions about models, training datasets, and the challenge datasets themselves.

Subfields: Artificial Intelligence, Infectious Diseases, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Infectious Diseases

In Evaluation & Benchmarks

Citations in the corpus

How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, background): “Fine-tuning on the benchmark may clue the model into such unintended correlations (Liu et al., 2019a).”
Pretrained Transformers Improve Out-of-Distribution Robustness (2020, methodology): “For STS-B, ReCoRD, and SST-2/IMDb, there is a noticeable drop in accuracy when testing on OOD examples.”
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, background): “Much recent work has made similar observations about the limitations of existing pretrained models (Jia and Liang, 2017; Naik et al., 2018; McCoy and Linzen, 2019; McCoy et al., 2019; Liu et al., 2019a,b).”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, methodology): “Future work could investigate the provenance of model weaknesses in more detail, for instance by using test cases from HATECHECK to “inoculate” training data (Liu et al., 2019).”
Avoiding catastrophic forgetting in mitigating model biases in sentence-pair classification with elastic weight consolidation (2020, methodology): “As RoBERTa (Liu et al., 2019b) has been shown to be more robust to adversarial training (Bartolo et al., 2020), we also include this in our evaluation.”
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? (2021, positive): “Now after several decades, thanks in part to pretrained neural language models (Liu et al., 2019b; Lewis et al., 2020; Raffel et al., 2020), we have high quality systems for many challenge tasks that seemed ?”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Yet, challenge sets may suffer from performance instability (Liu et al., 2019a; Rozen et al., 2019; Zhou et al., 2020) and often lack sufficient statistical power (Card et al.”
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning (2019, methodology): “The extent to which BERT QA does well on Q UOREF might indicate its capacity for coref-erential reasoning that was not exploited when it was trained on SQuAD (for a detailed discussion of this phenomenon, see Liu et al., 2019).”
Adversarial Filters of Dataset Biases (2020, methodology): “Given that these benchmarks are collected independently of the original SNLI task, the biases from SNLI are less likely to carry over; however these benchmarks might contain their own biases (Liu et al., 2019a).”
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension (2019, background): “As suggested Liu et al. (2019), if the model can generalize with a few examples from the new domain, poor performance on that domain is an indicator of a lack of training data diversity, rather than of fundamental model generalization weaknesses.”
Probing Natural Language Inference Models through Semantic Fragments (2019, methodology): “To study this, we introduce a novel variation of the inoculation through ﬁne-tuning methodology of Liu, Schwartz, and Smith (2019), which emphasizes maximizing the model’s aggregate score over multiple tasks (as opposed to only on challenge tasks).”
Diversify Your Datasets: Analyzing Generalization via Controlled Variance in Adversarial Datasets (2019, methodology): “We follow Liu et al. (2019a) and fine-tune the model on the phenomenon-specific examples, testing how many training examples the model needs to observe before it performs reasonably well on this inference type.6 We split each dataset to training (77”
DynaSent: A Dynamic Benchmark for Sentiment Analysis (2020, methodology): “Our Model 0 begins with the RoBERTa-base parameters (Liu et al., 2019b) and adds a three-way sentiment classiﬁer head.”
The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions (2020, methodology): “For all pre-trained transformer models, namely, BERT, RoBERTa, and XLNet, we use the same set of hyper-parameters for analysis consideration.”
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2019, methodology): “Standard inoculation (Liu et al., 2019a) is known to drop performance on the original task. We use a modified objective (Richardson et al., 2020) to alleviate this issue. Different from Talmor et al. (2019a), we find BERT and RoBERTa based QA models to be qualitatively similar, performing within 5% ”