A Closer Look at Memorization in Deep Networks

By Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron C. Courville, Yoshua Bengio and 1 more. 2017

2,230 citations recorded by the source corpus.

We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness. While deep networks are capable of memorizing noise data, our results suggest that they tend to prioritize learning simple patterns first. In our experiments, we expose qualitative differences in gradient-based optimization of deep neural networks (DNNs) on noise vs. real data. We also demonstrate that for appropriately tuned explicit regularization (e.g., dropout) we can degrade DNN training performance on noise datasets without compromising generalization on real data. Our analysis suggests that the notions of effective capacity which are dataset independent are unlikely to explain the generalization performance of deep networks when trained with gradient based methods because training data itself plays an important role in determining the degree of memorization.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Deep Learning Scaling is Predictable, Empirically (2017, background): “(1989); Blumer et al. (1989); Haussler et al. (1996)). All of these bounds show power-law relationships under certain assumptions, such as the hypothesis space must contain at least one model that can correctly fit the data or the training data size must be much larger than the capacity of the model”
Deduplicating Training Data Makes Language Models Better (2021, methodology): “In computer vision, memorization of training data has been studied from various angles for both discriminative and generative models (e.g. Arpit et al., 2017; Webster et al., 2019; Feldman and Zhang, 2020; Stephenson et al., 2021; Teter-wak et al., 2021).”
Shortcut learning in deep neural networks (2020, background): “The learning rate influences which patterns networks focus on: Large learning rates lead to learning simple patterns that are shared across examples, while small learning rates facilitate complex pattern learning and memorisation [116, 117].”
Are we done with ImageNet? (2020, background): “This is inspired by [40], which observes that long training schedules can be harmful in the presence of noisy data.”
mixup: Beyond Empirical Risk Minimization (2017, methodology): “Dropout (Srivastava et al., 2014) is considered the state-of-the-art method for learning with corrupted labels (Arpit et al., 2017).”
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, background): “5 łReasoningž in NLP There is still a debate(7) about if deep neural networks are capable of performing łreasoningž or just memorizing patterns based on large training data [3, 89].”
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data (2018, background): “The training objectives of overparameterized networks in principle have many (approximate) global optima and some generalize better than the others [14, 8, 2], while empirical observations imply that the optimization process in practice prefers those with better generalization.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “to capture a distribution, especially for some hard examples [21], [61], [148].”
Privacy in Deep Learning: A Survey (2020, background): “Deep neural networks have been shown to memorize the training data [46], rather than learning the latent properties of the data, which means they often tend to over-fit to the training data.”
Error Feedback Fixes SignSGD and other Gradient Compression Schemes (2019, background): “This ability of deep networks to generalize well on real data, while simultaneously being able to ﬁt arbitrary data has recently received a lot of attention (e.g. Soudry et al. [2018], Dinh et al. [2017], Zhang et al. [2018], Arpit et al. [2017], Kawaguchi et al. [2017]).”
On Robustness of Neural Architecture Search Under Label Noise (2020, background): “Great progress has been made in research on the robustness of learning algorithms under corrupted labels (Arpit et al., 2017; Chang et al., 2017; Ghosh et al., 2017; Patrini et al., 2017; Zhang et al., 2017, 2018; Jiang et al., 2018; Ren et al., 2018; Zhang and Sabuncu, 2018).”
Neural Additive Models: Interpretable Machine Learning with Neural Nets (2020, background): “This difficulty of learning large local fluctuations with ReLU networks without affecting their global behavior when fitting jagged functions might be due to their bias towards smoothness [2, 32].”
Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries (2022, background): “Unfortunately, maintaining the information diversity and purity is in trade-off ; clean examples mostly exhibit smaller losses (less diverse) than noise examples due to the memorization effect of deep neural networks [4, 33].”
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks (2019, background): “Deep Neural Networks can memorize details about particular instances, rather than only shared characteristics [29, 4].”