What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models

By Allyson Ettinger. 2019

680 citations recorded by the source corpus.

Pre-training by language modeling has become a popular and successful approach to NLP tasks, but we have yet to understand exactly what linguistic capacities these pre-training processes confer upon models. In this paper we introduce a suite of diagnostics drawn from human language experiments, which allow us to ask targeted questions about information used by language models for generating predictions in context. As a case study, we apply these diagnostics to the popular BERT model, finding that it can generally distinguish good from bad completions involving shared category or role reversal, albeit with less sensitivity than humans, and it robustly retrieves noun hypernyms, but it struggles with challenging inference and role-based event prediction— and, in particular, it shows clear insensitivity to the contextual impacts of negation.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020, neutral): “…from the same distribution as the ﬁne-tuning set, but from what we might term normative evaluation: expert-created controlled data sets that capture our intuitions about how an agent should perform the task (Marelli et al., 2014; Marvin and Linzen, 2018; Warstadt et al., 2019; Ettinger, 2020).”
Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2020, background): “, 2019), linguistic capacities (Ettinger, 2020; Kassner and Schtze, 2019), understanding of rare words (Schick and Schütze, 2020), and ability to perform symbolic reasoning (Talmor et al.”
AI and the Everything in the Whole Wide World Benchmark (2021, background): “, 2020], or the proper handling of negation [Ettinger, 2020]) nor present a truly varied range of ways to deploy linguistic knowledge in comprehension.”
HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “, 2017; Belinkov and Bisk, 2018) and language modelling (Marvin and Linzen, 2018; Ettinger, 2020).”
oLMpics-On What Language Model Pre-training Captures (2019, background): “…and fine-tuning a simple, often linear, randomly initialized probe, to determine whether the representations hold relevant information (Ettinger et al., 2016; Adi et al., 2016; Belinkov and Glass, 2019; Hewitt and Manning, 2019; Wallace et al., 2019; Rozen et al., 2019; Peters et al.,…”
Few-Shot Text Generation with Natural Language Instructions (2020, methodology): “The idea to rephrase tasks as cloze questions is commonly used to probe the knowledge contained within masked language models (e.g., Petroni et al., 2019; Wang et al., 2019; Talmor et al., 2020; Schick and Schütze, 2020; Ettinger, 2020; Kassner and Schütze, 2020; Sakaguchi et al., 2020).”
It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (2020, methodology): “It is also commonly used for probing the knowledge contained within LMs (Trinh and Le, 2018; Petroni et al., 2019; Talmor et al., 2019; Schick and Sch¨utze, 2020b; Ettinger, 2020, i.a. ).”
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, background): “Besides factual knowledge, large-scale pre-training also allows LMs to handle linguistic phenomena such as analogies [9], negations [25], semantic role sensitivity [25], semantic similarity [131], cant understanding [131], and rare word understanding [116].”
True Few-Shot Learning with Language Models (2021, methodology): “Prior work uses LMs for zero-shot learning by choosing an arbitrary prompt [17, 79] which requires no data but is suboptimal [4].”
Measuring and Improving Consistency in Pretrained Language Models (2021, background): “…(Rogers et al., 2020) perform on linguistic tasks (Goldberg, 2019; Hewitt and Man-ning, 2019; Tenney et al., 2019; Elazar et al., 2021), commonsense (Forbes et al., 2019; Da and Kasai, 2019; Zhang et al., 2020) and reasoning (Talmor et al., 2020; Kassner et al., 2020), usually assessed by measures…”
Dynabench: Rethinking Benchmarking in NLP (2021, background): “Challenging test sets have shown that many state-of-the-art NLP models struggle with compositionality (Nie et al., 2019; Kim and Linzen, 2020; Yu and Ettinger, 2020; White et al., 2020), and ﬁnd it difﬁcult to pass the myriad stress tests for social (Rudinger et al., 2018; May et al., 2019; Nangia…”
A Primer in BERTology: What We Know About How BERT Works (2020, background): “However, we do have evidence from an MLM probing study that BERT has some knowledge for semantic roles (Ettinger, 2019).”
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly (2019, background): “Ettinger (2019) defines and applies psycho-linguistic diagnostics for PLMs.”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, background): “At a high level, mechanisms for interpreting BERT can be categorized into three main categories: interpreting the learned embeddings [1, 16, 22, 47, 85], BERT’s learned knowledge of syntax [26, 29, 30, 43, 45, 74], and BERT’s learned knowledge of semantics [23, 74].”
Pre-Trained Models: Past, Present and Future (2021, background): “For the commonsense knowledge, Ettinger (2020) first evaluates PTMs ’ knowledge in the aspect of psycholinguists and find that the models perform well in the situation of shared category or role reversal but fail with challenging inferences and role-based event.”
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance (2019, background): “, 2019), or a variety of psycholinguistic phenomena (Ettinger, 2020); its internal workings (Coenen et al.”
Pre-trained models for natural language processing: A survey (2020, methodology): “zed the roles of BERT’s layers in dierent tasks and found that BERT solves tasks in a similar order to that in NLP pipelines. Furthermore, knowledge of subject-verb agreement [50] and semantic roles [44] are also conﬁrmed to exist in BERT. Besides, Hewitt and Manning [59], Jawahar et al. [72], Kim e”