What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties

By Alexis Conneau, Germán Kruszewski, Guillaume Lample, Loïc Barrault, Marco Baroni. 2018

1,024 citations recorded by the source corpus.

Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing. “Downstream” tasks, often based on sentence classification, are commonly used to evaluate the quality of sentence representations. The complexity of the tasks makes it however difficult to infer what kind of information is present in the representations. We introduce here 10 probing tasks designed to capture simple linguistic features of sentences, and we use them to study embeddings generated by three different encoders trained in eight distinct ways, uncovering intriguing properties of both encoders and training methods.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Neural Network Acceptability Judgments (2018, background): “, 2016), whether the subject is singular or plural (Conneau et al., 2018), or whether a given token is under the scope of negation (Ettinger et al.”
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (2019, background): “Outside NLI, multiple projects have used classification tasks to understand what linguistic and/or structural information is present in vector encodings of sentences (e.g., Adi et al., 2017; Ettinger et al., 2018; Conneau et al., 2018).”
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks (2018, methodology): “In the area of sentence-to-vector sentence encoding, Conneau et al. (2018) offer one of the most comprehensive suites of diagnostic tasks, 1 Quora Question Pairs and higlight the importance of ensuring that these models preserve lexical content information.”
Transfer Learning in Natural Language Processing (2019, background): “We will then shed some light on what the learned representations can and cannot capture based on recent studies (Conneau et al., 2018; Peters et al., 2018b).”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “The framework provides SentEval-style (Conneau et al., 2018) probing setup and datasets for each language available in Universal Dependencies (UD; Nivre et al.”
Locating and Editing Factual Associations in GPT (2022, background): “…are encoded in internal model representations, most commonly by training a probing classiﬁer to predict said properties from the representations (Ettinger et al., 2016; Adi et al., 2017; Hupkes et al., 2018; Conneau et al., 2018; Belinkov et al., 2017; Belinkov & Glass, 2019, inter alia).”
Fast Model Editing at Scale (2021, background): “structures (Belinkov et al., 2017; Conneau et al., 2018; Hewitt and Manning, 2019) or facts about the world (Petroni et al.”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, background): “One prominent methodology that attempts to shed light on those questions is probing (Conneau et al., 2018) (also known as auxilliary prediction [Adi et al.”
CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, methodology): “Structural methods try to identify the information encoded in the model’s internal structure by using its representations to classify textual properties (Adi et al. 2017; Hupkes, Veldhoen, and Zuidema 2018; Conneau et al. 2018).”
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias (2020, background): “Structural analyses aim to shed light on the internal structure of a neural model, for example through probing classiﬁers (Conneau et al., 2018; Hupkes et al., 2018; Adi et al., 2017) that predict linguistic properties using representations from trained models.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “Interestingly, Conneau et al. (2018) found that tasks requiring more nuanced linguistic knowledge (e.g., tree depth, coordination inversion) gain the most from using a deeper classiﬁer.”
What Does BERT Look at? An Analysis of BERT’s Attention (2019, background): “This kind of baseline is surprisingly strong at other probing tasks (Conneau et al., 2018).”
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models (2019, neutral): “Some approaches use fine-grained classification tasks to probe information in sentence embeddings (Adi et al., 2016; Conneau et al., 2018; Ettinger et al., 2018), or token-level and other sub-sentence level information in contextual embeddings (Tenney et al.”
Designing and Interpreting Probes with Control Tasks (2019, background): “, 2019), but multi-layer perceptron (MLP) probes are at least as popular (Belinkov et al., 2017; Conneau et al., 2018; Adi et al., 2017; Tenney et al., 2019; Ettinger et al., 2018).”
A Structural Probe for Finding Syntax in Word Representations (2019, neutral): “P ROJ 0 : Contextualizes the ELM O 0 embeddings with a randomly initialized BiLSTM layer of dimensionality identical to ELMo (1024), a surprisingly strong baseline for contextualization (Conneau et al., 2018).”