Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

By Alethea Power, Yuri Burda, Harrison Edwards, Igor Babuschkin, Vedant Misra. 2022

696 citations recorded by the source corpus.

In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. In some situations we show that neural networks learn through a process of"grokking"a pattern in the data, improving generalization performance from random chance level to perfect generalization, and that this improvement in generalization can happen well past the point of overfitting. We also study generalization as a function of dataset size and find that smaller datasets require increasing amounts of optimization for generalization. We argue that these datasets provide a fertile ground for studying a poorly understood aspect of deep learning: generalization of overparametrized neural networks beyond memorization of the finite training dataset.

Subfields: Artificial Intelligence, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Scaling & Learning Theory

Citations in the corpus

Predictability and Surprise in Large Generative Models (2022, background): “Similar behavior has also been observed during the training process of an individual model (rather than as a function of model size) for algorithmic tasks, and has been termed “grokking” [52].”
Unsolved Problems in ML Safety (2021, background): “Others have observed instances where a model’s training loss remains steady, but then its test performance spontaneously ascends from random chance to perfect generalization [141].”
An Explanation of In-context Learning as Implicit Bayesian Inference (2021, neutral): “This may be related to phenomena from overparameterization and overtraining (Zhang et al., 2017; Power et al., 2021).”
Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks (2022, background): “While this deﬁnition is somehow arbitrary, in practice we observed a grokking phenomenon (Power et al., 2021) where very soon after the accuracy became higher than random level it also became almost perfect (accuracy > 95% ).”
Faith and Fate: Limits of Transformers on Compositionality (2023, background): “We explore whether extended training beyond overfitting leads to improved generalization abilities, a phenomenon known as grokking [59, 53].”
RAFT: Adapting Language Model to Domain Specific RAG (2024, background): “( Brown et al., 2020; Power et al., 2022; Liu et al., 2022b) further explored how memorization impacts the models’ generalization capabilities.”
Large Language Models (2023, background): “In [9, 110] it was argued that these two metrics improve at two distinct stages of training.”
Training Trajectories of Language Models Across Scales (2022, background): “The main finding is that there exists a critical phase chage (Power et al., 2022; Nanda and Lieberum, 2022) that forms the incontext learning ability.”
Emergent and Predictable Memorization in Large Language Models (2023, background): “…(to the best of our knowledge), some forms of memorization are actually beneficial: we want large language models to memorize factual events and details to avoid “hallucinating” plausible-sounding but errant facts to unsuspecting users [Power et al., 2022, Cao et al., 2022, Tirumala et al., 2022b].”
Progress measures for grokking via mechanistic interpretability (2023, methodology): “In contrast to Power et al. (2022) but in line with Liu et al. (2022), grokking does not occur when both train and test loss improve together without the initial divergence, as shown in many of the figures in this paper, for example Figures 2 and 18.”
Mechanistic Interpretability for AI Safety - A Review (2024, background): “The work by Varma et al. (2023) builds on the work that analyzes transformers trained on modular addition (Nanda et al., 2023a) and explains grokking in terms of circuit efficiency, illustrating how a comprehensive understanding of a toy model can enable interesting analyses on top of that…”