On the geometry of generalization and memorization in deep neural networks

By Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, SueYeon Chung. 2021

99 citations recorded by the source corpus.

Understanding how large neural networks avoid memorizing training data is key to explaining their high generalization performance. To examine the structure of when and where memorization occurs in a deep network, we use a recently developed replica-based mean field theoretic geometric analysis method. We find that all layers preferentially learn from examples which share features, and link this behavior to generalization performance. Memorization predominately occurs in the deeper layers, due to decreasing object manifolds' radius and dimension, whereas early layers are minimally affected. This predicts that generalization can be restored by reverting the final few layer weights to earlier epochs before significant memorization occurred, which is confirmed by the experiments. Additionally, by studying generalization under different model sizes, we reveal the connection between the double descent phenomenon and the underlying model geometry. Finally, analytical analysis shows that networks avoid memorization early in training because close to initialization, the gradient contribution from permuted examples are small. These findings provide quantitative evidence for the structure of memorization across layers of a deep neural network, the drivers for such structure, and its connection to manifold geometric properties.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Scaling & Learning Theory

Citations in the corpus

Deduplicating Training Data Makes Language Models Better (2021, methodology): “In computer vision, memorization of training data has been studied from various angles for both discriminative and generative models (e.g. Arpit et al., 2017; Webster et al., 2019; Feldman and Zhang, 2020; Stephenson et al., 2021; Teter-wak et al., 2021).”
Editing Factual Knowledge in Language Models (2021, background): “Approaches that incentivize models to memorize factual knowledge show to be beneficial for many downstream tasks suggesting that research on methods that effectively edit the memory of a model is indeed important (Zhang et al., 2019; Sun et al., 2019, 2020).”
Task-Aware Information Routing from Common Representation Space in Lifelong Learning (2023, background): “The intuition behind placing TAMs higher up in the layer hierarchy is as follows: the early layers of DNNs capture generic information, while the later layers memorize due to the diminishing dimension and radius of the manifold (Stephenson et al., 2020; Baldock et al., 2021).”
Deep Learning Through the Lens of Example Difficulty (2021, background): “Early layers generalize while later layers memorize (Stephenson et al., 2021).”
Sparse Prototype Code Underlies Classification and Prediction Across Modalities (2026, unknown): “Predictive theories of representational geometry —manifold capacity [Chung et al., 2018, Cohen et al., 2020, Stephenson et al., 2021, Wakhloo et al., 2023] and neural collapse [Papyan et al., 2020, Han et al., 2021, Galanti et al., 2021] — have made important progress in connecting geometry to…”