oLMpics-On What Language Model Pre-training Captures

By Alon Talmor, Yanai Elazar, Yoav Goldberg, Jonathan Berant. 2019

316 citations recorded by the source corpus.

Abstract Recent success of pre-trained language models (LMs) has spurred widespread interest in the language capabilities that they possess. However, efforts to understand whether LM representations are useful for symbolic reasoning tasks have been limited and scattered. In this work, we propose eight reasoning tasks, which conceptually require operations such as comparison, conjunction, and composition. A fundamental challenge is to understand whether the performance of a LM on a task should be attributed to the pre-trained representations or to the process of fine-tuning on the task data. To address this, we propose an evaluation protocol that includes both zero-shot evaluation (no fine-tuning), as well as comparing the learning curve of a fine-tuned LM to the learning curve of multiple controls, which paints a rich picture of the LM capabilities. Our main findings are that: (a) different LMs exhibit qualitatively different reasoning abilities, e.g., RoBERTa succeeds in reasoning tasks where BERT fails completely; (b) LMs do not reason in an abstract manner and are context-dependent, e.g., while RoBERTa can compare ages, it can do so only when the ages are in the typical range of human ages; (c) On half of our reasoning tasks all models fail completely. Our findings and infrastructure can help future work on designing new datasets, models, and objective functions for pre-training.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Evaluation & Benchmarks

Citations in the corpus

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering (2020, background): “Recently, several works have shown that factual information can be extracted from large scale language models trained on vast quantities of data (Radford et al., 2019; Petroni et al., 2019; Jiang et al., 2019; Talmor et al., 2019).”
How Much Knowledge Can You Pack into the Parameters of a Language Model? (2020, background): “…“language models as knowledge bases” has typically tried to understand the scope of the information stored in the model using synthetic tasks that are similar to the pre-training objective (Petroni et al., 2019; Jiang et al., 2019) and/or measure reasoning capabilities (Talmor et al., 2019).”
Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2020, background): “, 2019), probing the understanding of rare words (Schick and Schütze, 2019), and investigating a language model’s ability to perform symbolic reasoning (Talmor et al., 2019).”
How Context Affects Language Models' Factual Predictions (2020, methodology): “As a community, we should continue to strive for greater diversity in our criteria and possible use-cases for assessing such models [Talmor et al., 2019].”
Making Pre-trained Language Models Better Few-shot Learners (2021, background): “Finally, there is a large body of work on prompting for mining knowledge from pre-trained models (Trinh and Le, 2018; Petroni et al., 2019; Davison et al., 2019; Talmor et al., 2020, inter alia).”
Few-Shot Text Generation with Natural Language Instructions (2020, methodology): “The idea to rephrase tasks as cloze questions is commonly used to probe the knowledge contained within masked language models (e.g., Petroni et al., 2019; Wang et al., 2019; Talmor et al., 2020; Schick and Schütze, 2020; Ettinger, 2020; Kassner and Schütze, 2020; Sakaguchi et al., 2020).”
RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, background): “PLMs have been shown to have some reasoning capabilities (Talmor et al., 2020b), but fail on basic reasoning tasks (Talmor et al., 2020a) and are inconsistent (Elazar et al., 2021), especially when it comes to negation (Kassner and Schütze, 2020).”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, neutral): “Our work joins a line of previously introduced benchmarks targeting commonsense reasoning 365 [12, 13, 29, 15] as well as probing for other reasoning abilities [32, 33, 34, 35, 36, 37, 38].”
Transformers as Soft Reasoners over Language (2020, background): “There have been numerous other demonstrations that transformers either already know [Talmor et al., 2019; Richardson and Sabharwal, 2019] or can learn to emulate other algorithms, including for semantic parsing [He and Choi, 2019], machine translation [Wang et al.”
Inner Monologue: Embodied Reasoning through Planning with Language Models (2022, background): “One of the remarkable observations in recent machine learning research is that large language models (LLMs) can not only generate fluent textual descriptions, but also appear to have rich internalized knowledge about the world [3, 4, 5, 6, 7].”
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2022, background): ““Dante was born in 〈PLACE〉”) can be extracted [31, 18, 9, 43, 36].”
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts (2021, background): “, 2019), reasoning (Talmor et al., 2020; Brown et al., 2020), summarization (Radford et al.”
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, methodology): “And the cloze test task is served as a probing task, which has been widely used in NLP [50, 68].”
CodeBERT: A Pre-Trained Model for Programming and Natural Languages (2020, methodology): “, 2019), and another one has fewer candidates which are filter or curated based on experts’ understanding about the ability to be tested (Talmor et al., 2019).”
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2020, background): “, 2020), reasoning (Talmor et al., 2019), and common sense (Forbes et al.”
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, background): “Talmor et al. (2019) ﬁnd that language models fail completely on half of eight reasoning tasks that require symbolic operations such as comparison, conjunction, and composition.”
Information-Theoretic Probing with Minimum Description Length (2020, methodology): “…to task performance, learning curves have also been used before by Yogatama et al. (2019) to evaluate how quickly a model learns a new task, and by Talmor et al. (2019) to understand whether the performance of a LM on a task should be attributed to the pre-trained representations or to the…”