Investigating BERT’s Knowledge of Language: Five Analysis Methods with NPIs

By Alex Warstadt, Yuning Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish and 6 more. 2019

133 citations recorded by the source corpus.

Though state-of-the-art sentence representation models can perform tasks requiring significant knowledge of grammar, it is an open question how best to evaluate their grammatical knowledge. We explore five experimental methods inspired by prior work evaluating pretrained sentence representation models. We use a single linguistic phenomenon, negative polarity item (NPI) licensing, as a case study for our experiments. NPIs like any are grammatical only if they appear in a licensing environment like negation (Sue doesn’t have any cats vs. *Sue has any cats). This phenomenon is challenging because of the variety of NPI licensing environments that exist. We introduce an artificially generated dataset that manipulates key features of NPI licensing for the experiments. We find that BERT has significant knowledge of these features, but its success varies widely across different experimental methods. We conclude that a variety of methods is necessary to reveal all relevant aspects of a model’s grammatical knowledge in a given domain.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

How Context Affects Language Models' Factual Predictions (2020, background): “…such as syntax trees [Marvin and Linzen, 2018, Hewitt and Manning, 2019, Vig and Belinkov, 2019], negative polarity items [Warstadt and Bowman, 2019, Warstadt et al., 2019], semantic fragments [Richardson et al., 2019], function words [Kim et al., 2019], and many other linguistic phenomena…”
oLMpics-On What Language Model Pre-training Captures (2019, background): “…simple, often linear, randomly initialized probe, to determine whether the representations hold relevant information (Ettinger et al., 2016; Adi et al., 2016; Belinkov and Glass, 2019; Hewitt and Manning, 2019; Wallace et al., 2019; Rozen et al., 2019; Peters et al., 2018b; Warstadt et al., 2019).”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, background): “These works cover a wide variety of topics, including grammatical generalization (Goldberg, 2019; Warstadt et al., 2019), syntax (Tenney et al.”
A Primer in BERTology: What We Know About How BERT Works (2020, background): “Furthermore, different probing methods may reveal complementary or even contradictory information, in which case a single test (as done in most studies) would not be sufﬁcient (Warstadt et al., 2019).”
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly (2019, background): “Warstadt et al. (2019) investigate BERT’s understanding of how negative polarity items are licensed.”
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models (2020, methodology): “Early versions of jiant have already been used in multiple works, including probing analyses (Ten-ney et al., 2019b,a; Warstadt et al., 2019; Lin et al., 2019; Hewitt and Manning, 2019; Jawahar et al., 2019), transfer learning experiments (Wang et al., 2019a; Phang et al., 2018), and dataset and…”
BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance (2019, background): “…(Ab-nar et al., 2019); its handling of speciﬁc linguistic phenomena such as subject-verb agreement (Goldberg, 2019), negative polarity items (Warstadt et al., 2019), function words (Kim et al., 2019), or a variety of psycholinguistic phenomena (Ettinger, 2019); or its internal workings…”
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020, methodology): “…2018), identifying speciﬁc knowledge or phenomena encapsulated in the representations learned by BERT using probing tasks (Tenney et al., 2019b,a; Warstadt et al., 2019a; Lin et al., 2019a; Hewitt and Manning, 2019; Jawahar et al., 2019), analyzing the attention heads of BERT (Clark et al.,…”
Do Attention Heads in BERT Track Syntactic Dependencies? (2019, methodology): “We run experiments on the uncased BERT-large model ﬁne-tuned on the Corpus of Linguistic Acceptability (CoLA; Warstadt et al., 2019b) and MNLI to investigate the impact of ﬁne-tuning on a syntax-related task (CoLA) and a semantic-related task (MNLI) on the structure of attention weights and…”
Probing Natural Language Inference Models through Semantic Fragments (2019, background): “…and small subsets of language, which includes work on syntactic probing (McCoy, Pavlick, and Linzen 2019; Goldberg 2019), probing basic reasoning (We-ston et al. 2015; Geiger et al. 2018; 2019) and probing other tasks (Lake and Baroni 2017; Chrupała and Alishahi 2019; Warstadt et al. 2019).”
Negated LAMA: Birds cannot fly (2019, background): “There is a wide range of literature analyzing linguistic knowledge stored in pretrained embed-dings (Jumelet and Hupkes, 2018; Gulordava et al., 2018; Giulianelli et al., 2018; McCoy et al., 2019; Dasgupta et al., 2018; Martin and Linzen, 2018; Warstadt and Bowman, 2019; Kann et al., 2019).”
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT (2020, background): “Much recent work focuses on the grammatical and syntactic capabilities of BERT (Hewitt and Manning, 2019; Liu et al., 2019; Swayamdipta et al., 2019; Goldberg, 2019; Wolf, 2019; Coenen et al., 2019; Tenney et al., 2019; Warstadt et al., 2019; Kim et al., 2019).”
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge (2019, background): “This shows the high capacity for transformer models to absorb new tasks with minimal costs, as also observed in Richardson et al. (2020) for NLI.”