Probing for semantic evidence of composition by means of simple classification tasks

By Allyson Ettinger, Ahmed Elgohary, Philip Resnik. 2016

168 citations recorded by the source corpus.

We propose a diagnostic method for probing specific information captured in vector representations of sentence meaning, via simple classification tasks with strategically constructed sentence sets. We identify some key types of semantic information that we might expect to be captured in sentence composition, and illustrate example classification tasks for targeting this information.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

SentEval: An Evaluation Toolkit for Universal Sentence Representations (2018, background): “On the other hand, it has been argued that the focus should be on downstream tasks where these representations would actually be applied (Ettinger et al., 2016; Nayak et al., 2016).”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “Probing has emerged as a significant evaluation paradigm to analyze and interpret the inner workings of LLMs (Ettinger et al., 2016; Adi et al., 2017; Belinkov et al., 2017; Hupkes et al., 2018; Tenney et al., 2018; Belinkov and Glass, 2019; Teehan et al., 2022), although it comes with certain…”
Analysis Methods in Neural Language Processing: A Survey (2018, methodology): “) Classification; also other methods (Ettinger et al., 2016) Sentence embeddings Semantic role, word presence Classification (Adi et al.”
Designing and Interpreting Probes with Control Tasks (2019, background): “Early probing papers used linear functions (Shi et al., 2016; Ettinger et al., 2016; Alain and Bengio, 2016), which are still used (Bisazza and Tump, 2018; Liu et al., 2019), but multi-layer perceptron (MLP) probes are at least as popular (Belinkov et al., 2017; Conneau et al., 2018; Adi et al.,…”
Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task (2017, background): “, 2015) and what linguistic information is encoded in models’ produced representations (Adi et al., 2016; Ettinger et al., 2016).”
Pre-Trained Models: Past, Present and Future (2021, methodology): “Representation probing have been widely applied to analyze NLP neural models from word embeddings to PTMs (Köhn, 2015; Ettinger et al., 2016; Shi et al., 2016; Adi et al., 2017; Conneau et al., 2018a; Hewitt and Manning, 2019; Glavaš and Vulić, 2021).”
Investigating BERT’s Knowledge of Language: Five Analysis Methods with NPIs (2019, methodology): “For example, Shi et al. (2016), Ettinger et al. (2016), and Tenney et al. (2019) use probing tasks to target a model’s knowledge of particular grammatical features.”
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension (2019, methodology): “Ettinger et al. (2016, 2017); Zhu et al. (2018), i.a., use a task-based approach similar to ours, where tasks that require a speciﬁc subset of linguistic knowledge are used to perform qualitative evaluation.”
When Do You Need Billions of Words of Pretraining Data? (2020, methodology): “We use the widely-adopted probing approach of Ettinger et al. (2016), Adi et al. (2017), and others— which we call classiﬁer probing —to test the extent to which linguistic features like part-of-speech and coreference are encoded in the MiniBERTa representations.”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “ts/statements - Transparency High-standard documentation All - Technical explainability Factsheets (explainability) Saliency maps [61], self-attention patterns [40], influence functions [38], probing [16] Adaptable user-centered explainability Factsheets (explainability) Counterfactual [21], contras”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, background): “eludes word embeddings [138, 188, 279], RNN hidden states or gate activations [102, 278, 306, 335, 345], and sentence embeddings [2, 3, 47, 81, 106, 117].”
Similarity Analysis of Contextual Word Representation Models (2020, background): “The most common approach for analyzing neural network models in general, and contextual word representations in particular, is by probing classifiers (Ettinger et al., 2016; Belinkov et al., 2017; Adi et al., 2017; Conneau et al., 2018; Hupkes et al., 2018), where a classifier is trained on a corpus”
Do Language Embeddings capture Scales? (2020, methodology): “Probes We use linear probes in all cases following many previous probing work (Shi et al., 2016; Ettinger et al., 2016; Pimentel et al., 2020) since we want to use a simple probe to find easily accessible information in a representation.”
Can neural networks acquire a structural bias from raw linguistic data? (2020, methodology): “Data Generation The data for these experiments are automatically generated using a method similar to that adopted by Ettinger, Elgohary, and Resnik (2016) and Marvin and Linzen (2018).”