Frequency Effects on Syntactic Rule Learning in Transformers

By Jason Wei, Dan Garrette, Tal Linzen, Ellie Pavlick. 2021

81 citations recorded by the source corpus.

Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules. We investigate this question using the case study of BERT’s performance on English subject–verb agreement. Unlike prior work, we train multiple instances of BERT from scratch, allowing us to perform a series of controlled interventions at pre-training time. We show that BERT often generalizes well to subject–verb pairs that never occurred in training, suggesting a degree of rule-governed behavior. We also find, however, that performance is heavily influenced by word frequency, with experiments showing that both the absolute frequency of a verb form, as well as the frequency relative to the alternate inflection, are causally implicated in the predictions BERT makes at inference time. Closer analysis of these frequency effects reveals that BERT’s behavior is consistent with a system that correctly applies the SVA rule in general but struggles to overcome strong training priors and to estimate agreement features (singular vs. plural) on infrequent lexical items.

Subfields: Artificial Intelligence, Model Architectures, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Computational Linguistics & Semantics

Citations in the corpus

Emergent Abilities of Large Language Models (2022, neutral): “Overall, more work is needed to tease apart what enables scale to unlock emergent abilities.”
Impact of Pretraining Term Frequencies on Few-Shot Reasoning (2022, methodology): “Impact of Frequency on LM Performance Kassner et al. (2020) and Wei et al. (2021) perform controlled experiments varying pretraining data to characterize the extent pretraining affects LMs’ ability to learn to memorize and reason with facts as well as learn generalizable syntax rules.”
Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions (2022, background): “Finally, Wei et al. (2021) investigated how word frequency and co-occurrences causally affect subject–verb agreement.”
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2021, background): “Though many of the abstractions that we study have been discussed in prior analyses of LMs, the only one for which prior work has enforced novelty is subject-verb agreement (Wei et al., 2021).”
Large Language Models Struggle to Learn Long-Tail Knowledge (2022, background): “Other work notices a correspondence between model accuracy and data frequency for different knowledge-intensive tasks (Petroni et al., 2019; Kassner et al., 2020; De Cao et al., 2021; Wei et al., 2021).”