What do you learn from context? Probing for sentence structure in contextualized word representations

By Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das and 1 more. 2019

968 citations recorded by the source corpus.

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. Building on recent token-level probing work, we introduce a novel edge probing task design and construct a broad suite of sub-sentence tasks derived from the traditional structured NLP pipeline. We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for syntactic phenomena, but only offer comparably small improvements on semantic tasks over a non-contextual baseline.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, methodology): “For WSC, which is a span-based task, we use a model inspired by Tenney et al. (2019).”
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (2018, background): “The successes of sentence encoder pretraining have sparked a line of work analyzing these models (Zhang and Bowman, 2018; Peters et al., 2018b; Tenney et al., 2019b; Peters et al., 2019; Tenney et al., 2019a; Liu et al., 2019, i.a.).”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public.”
How Context Affects Language Models' Factual Predictions (2020, background): “…2018, Hewitt and Manning, 2019, Vig and Belinkov, 2019], negative polarity items [Warstadt and Bowman, 2019, Warstadt et al., 2019], semantic fragments [Richardson et al., 2019], function words [Kim et al., 2019], and many other linguistic phenomena [Tenney et al., 2019a,b, De Cao et al., 2020].”
oLMpics-On What Language Model Pre-training Captures (2019, background): “This has instigated research exploring what is captured by the contextualized representations that these LMs compute, revealing that they encode substantial amounts of syntax and semantics (Linzen et al., 2016b; Tenney et al., 2019b,a; Shwartz and Dagan, 2019; Lin et al., 2019; Coenen et al., 2019).”
Measuring Bias in Contextualized Word Representations (2019, methodology): “Model We use the model suggested on Kaggle, 6 inspired by Tenney et al. (2019).”
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models (2021, neutral): “We suspect that adverbs are acting as cues that a verb will soon appear, thus increasing the probability of both the correct and incorrect verb, but increasing that of the correct verb more (for similar findings in human sentence processing, see (Vasishth and Lewis, 2006)).”
Factual Probing Is [MASK]: Learning vs. Learning to Recall (2021, background): “…in linguistic probing, which aims to explore the linguistic properties encoded in contextualized word representations (Belinkov et al., 2017; Tenney et al., 2019; Lin et al., 2019)—for example, by seeing if a classifier can predict that “chef ” is the nominal subject of “made” given the…”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, methodology): “Finally, we note that our approach performs an ablation of these properties in the representation space, which reveals which layers are actually responsible for processing properties, as opposed to Tenney et al. (2019a), who focused on where this information is easily extractable.”
What Does BERT Look at? An Analysis of BERT’s Attention (2019, neutral): “Like our work, some of these studies have also demonstrated models capturing aspects of syntax (Shi et al., 2016; Blevins et al., 2018) or coreference (Tenney et al., 2018, 2019; Liu et al., 2019) without explicitly being trained for the tasks.”
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (2020, background): “…agreement (Goldberg, 2019; Jawahar et al., 2019), constituent types, dependency labels, NER, and (core) semantic role types (again, all in English) (Tenney et al., 2019).4 Hewitt and Manning (2019) find information analogous to unlabeled dependency structures in the word vectors provided by ELMo…”