An Empirical Study of Example Forgetting during Deep Neural Network Learning

By Mariya Toneva, Alessandro Sordoni, Rémi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoffrey J. Gordon. 2018

1,017 citations recorded by the source corpus.

Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a “forgetting event” to have occurred when an individual training example transitions from being classified correctly to incorrectly over the course of learning. Across several benchmark data sets, we find that: (i) certain examples are forgotten with high frequency, and some not at all; (ii) a data set’s (un)forgettable examples generalize across neural architectures; and (iii) based on forgetting dynamics, a significant fraction of examples can be omitted from the training data set while still maintaining state-of-the-art generalization performance.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

On The Impact of Machine Learning Randomness on Group Fairness (2023, background): “The changing predictive behavior of neural networks beyond training loss convergence is not surprising, and has been studied extensively in literature [29, 32, 54, 55].”
Understanding Dataset Difficulty with V-Usable Information (2021, neutral): “Estimates relying on model training dynamics (Toneva et al., 2018; Swayamdipta et al., 2020), gradient magnitudes (Vodra-halli et al., 2018), or loss magnitudes (Han et al., 2018) are sensitive to factors such as variance during steps of training.”
The Right Tool for the Job: Matching Model and Instance Complexities (2020, methodology): “It also allows for controlling the speed/accuracy tradeoff using a single model, without retraining it for any point along the curve.”
Wide Neural Networks Forget Less Catastrophically (2021, background): “Alleviating catastrophic forgetting has attracted a lot of attention lately, and many interesting solutions are proposed to partly overcome the issue (e.g., Toneva et al., 2019; Nguyen et al., 2019; Hsu et al., 2018; Li et al., 2019; Wallingford et al., 2020).”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “They also play a role in multi-task learning, re-inforcement learning and standard supervised learning, for example under distribution shift (Arivazhagan et al., 2019; Toneva et al., 2018).”
Linear Mode Connectivity in Multitask and Continual Learning (2020, background): “A growing body of research has attempted to tackle this problem in recent years (e.g Parisi et al., 2018; Toneva et al., 2018; Nguyen et al., 2019; Farajtabar et al., 2019; Hsu et al., 2018; Rusu et al., 2016; Li et al., 2019; Kirkpatrick et al., 2017; Zenke et al., 2017; Shin et al., 2017; Rolnick ”
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix (2020, background): “Recently, a lot of eﬀorts has been put toward dissecting CF [Toneva et al., 2018].”
Efficient Meta Lifelong-Learning with Limited Memory (2020, background): ““Forgettable” picks examples according to forgetting events (Toneva et al., 2019).”
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics (2020, background): “[8] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J.”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, methodology): “In light of recent work on designing alternative replay strategies [2, 7, 40], we hypothesize that “interfering” examples (i.”
Understanding the Role of Training Regimes in Continual Learning (2020, background): “A growing body of research has attempted to tackle this problem in recent years [58, 72, 57, 29].”
Orthogonal Gradient Descent for Continual Learning (2019, background): “There is a growing interest in measuring catastrophic forgetting (Toneva et al., 2018; Kemker et al., 2018), evaluating continual learning algorithms (Farquhar and Gal, 2018; Hayes et al.”
Episodic Memory in Lifelong Language Learning (2019, methodology): “We ﬁnd that this is a strong baseline that outperforms other simple methods based on surprisal (Ramalho & Garnelo, 2019) and the concept of forgettable examples (Toneva et al., 2019) in our preliminary experiments.”
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020, methodology): “We hypothesize that hardto-learn examples—those with low confidence— might be mislabeled, as has also been suggested in prior work (Manning, 2011; Toneva et al., 2018).”
Information-Theoretic Measures of Dataset Difficulty (2021, background): “Estimates relying on model training dynamics (Swayamdipta et al., 2020; Toneva et al., 2018), gradient magnitudes (Vodrahalli et al., 2018), or loss magnitudes (Han et al., 2018) are sensitive to factors such as variance due to initial parameterization.”
Dataset Condensation with Gradient Matching (2020, methodology): “[16] (a) (b) Figure 1: Dataset Condensation (left) aims to generate a small set of synthetic images that can match the performance of a network trained on a large image dataset.”
SemDeDup: Data-efficient learning at web-scale through semantic deduplication (2023, background): “Recent approaches to achieve data efﬁciency in deep learning have operated in a supervised setting by deﬁning and ﬁnding “hard” examples not easily learned by partially or fully trained (ensembles of) models [27, 28, 29, 15, 30].”
Meta-Learning with Sparse Experience Replay for Lifelong Language Learning (2020, methodology): “A low A-GEM performance compared to a simple replay method on this benchmark was also observed in Wang et al. (2019). OML-ER and ANML-ER significantly outperform all the baselines, and the former achieves the highest accuracy overall but, again, the differences between the two are not statistically ”