Are all training examples equally valuable?

By Àgata Lapedriza, Hamed Pirsiavash, Zoya Bylinskii, Antonio Torralba. 2013

83 citations recorded by the source corpus.

When learning a new concept, not all training examples may prove equally useful for training: some may have higher or lower training value than others. The goal of this paper is to bring to the attention of the vision community the following considerations: (1) some examples are better than others for training detectors or classifiers, and (2) in the presence of better examples, some examples may negatively impact performance and removing them may be beneficial. In this paper, we propose an approach for measuring the training value of an example, and use it for ranking and greedily sorting examples. We test our methods on different vision tasks, models, datasets and classifiers. Our experiments show that the performance of current state-of-the-art detectors and classifiers can be improved when training on a subset, rather than the whole training set.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Dataset Distillation (2018, methodology): “…is to summarize the entire dataset by a small subset, either by only using the “valuable” data for model training (Angelova et al., 2005; Felzenszwalb et al., 2010; Lapedriza et al., 2013) or by only labeling the “valuable” data via active learning (Cohn et al., 1996; Tong & Koller, 2001).”
Self-Paced Learning with Diversity (2014, methodology): “A straightforward way to design a curriculum is to select samples based on certain heuristical " easiness " measurements [3] [4] [5].”
Flexible Dataset Distillation: Learn Labels Instead of Images (2020, background): “In this sense it is related to other endeavours such as dataset pruning [2, 10, 16], core-set construction [29, 3, 26] and instance selection [22] that focuses on dataset summarization through a small number of examples.”
Deep Q-Networks for Accelerating the Training of Deep Neural Networks (2016, background): “Training samples are not equally valuable in terms of the contribution to the generalization performance [10].”
What Objective Does Self-paced Learning Indeed Optimize? (2015, methodology): “Instead of heuristically designing a curriculum by ranking samples based on manually preset easiness measurements as CL presented [4, 5], the SPL research further attempted to formulate this ad-hoc idea as a concise SPL model through introducing a regularization term into the learning objective.”
MentorNet: Regularizing Very Deep Neural Networks on Corrupted Labels (2017, background): “In parallel to curriculum learning, a variety of weighting schemes, which assign predefined weights to training examples, have been studied in the computer vision community [10, 23, 31, 27, 46].”
Dataset Distillation by Matching Training Trajectories (2022, background): “Related to coreset, other lines of research aim to understand which training samples are “valuable” for modern machine learning, including measuring single-example accuracy [20] and counting misclassiﬁcation rates [39].”