Are Pretrained Language Models Symbolic Reasoners over Knowledge?

By Nora Kassner, Benno Krojer, Hinrich Schütze. 2020

79 citations recorded by the source corpus.

How can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work has attempted to quantify the number of facts PLMs learn, but we present, using synthetic data, the first study that investigates the causal relation between facts present in training and facts learned by the PLM. For reasoning, we show that PLMs seem to learn to apply some symbolic reasoning rules correctly but struggle with others, including two-hop reasoning. Further analysis suggests that even the application of learned reasoning rules is flawed. For memorization, we identify schema conformity (facts systematically supported by other facts) and frequency as key factors for its success.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021, background): “While there have been initial attempts to teach reasoning with rules to PLMs (Clark et al., 2020; Kassner et al., 2020), such approaches model only a subclass of logical rules.”
Measuring and Improving Consistency in Pretrained Language Models (2021, background): “…2020) perform on linguistic tasks (Goldberg, 2019; Hewitt and Man-ning, 2019; Tenney et al., 2019; Elazar et al., 2021), commonsense (Forbes et al., 2019; Da and Kasai, 2019; Zhang et al., 2020) and reasoning (Talmor et al., 2020; Kassner et al., 2020), usually assessed by measures of accuracy.”
Impact of Pretraining Term Frequencies on Few-Shot Reasoning (2022, methodology): “Multiplication Performance: Plot of GPT-J-6B’s 2-shot accuracy on multiplication (averaged over multiple multi-plicands and training instances) against the frequency of the equa-tion’s ﬁrst term in the pretraining corpus.”
Large Language Models Struggle to Learn Long-Tail Knowledge (2022, background): “Other work notices a correspondence between model accuracy and data frequency for different knowledge-intensive tasks (Petroni et al., 2019; Kassner et al., 2020; De Cao et al., 2021; Wei et al., 2021; Févry et al., 2020) and for domains outside of NLP (Rao et al.”
Critical Thinking for Language Models (2020, background): “Training on synthetic knowledge-graph data (such as "Paris CapitalOf France" and "France HasCapital Paris") from scratch, Kassner et al. (2020) find that BERT is able to correctly infer novel facts.”
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022, background): “With the recent success of large-scale pre-trained language models (PLMs), there has been growing interest in investigating what is captured by these PLMs during pre-training (Talmor et al., 2020a; Rogers et al., 2020; Kassner et al., 2020).”
Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions (2022, background): “, 2021a), negation (Ettinger, 2020; Kassner and Schütze, 2020), multilinguality (Jiang et al., 2020; Kassner et al., 2020), singular and plural hypernymy probes (Ravichander et al.”
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language (2022, background): “It has been shown that finetuning LMs on logical reasoning tasks makes them learn spurious correlations (Zhang et al., 2022b; Schlegel et al., 2022), and struggle with multi-hop reasoning (Kassner et al., 2020).”
History Compression via Language Models in Reinforcement Learning (2022, background): “As shown by Petroni et al. (2019); Talmor et al. (2020); Kassner et al. (2020), PLTs can learn abstract symbolic rules and perform reasoning.”