Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks

By Curtis G. Northcutt, Anish Athalye, Jonas W. Mueller. NeurIPS Datasets and Benchmarks, 2021

711 citations recorded by the source corpus.

We identify label errors in the test sets of 10 of the most commonly-used computer vision, natural language, and audio datasets, and subsequently study the potential for these label errors to affect benchmark results. Errors in test sets are numerous and widespread: we estimate an average of at least 3.3% errors across the 10 datasets, where for example label errors comprise at least 6% of the ImageNet validation set. Putative label errors are identified using confident learning algorithms and then human-validated via crowdsourcing (51% of the algorithmically-flagged candidates are indeed erroneously labeled, on average across the datasets). Traditionally, machine learning practitioners choose which model to deploy based on test accuracy - our findings advise caution here, proposing that judging models over correctly labeled test sets may be more useful, especially for noisy real-world datasets. Surprisingly, we find that lower capacity models may be practically more useful than higher capacity models in real-world datasets with high proportions of erroneously labeled data. For example, on ImageNet with corrected labels: ResNet-18 outperforms ResNet-50 if the prevalence of originally mislabeled test examples increases by just 6%. On CIFAR-10 with corrected labels: VGG-11 outperforms VGG-19 if the prevalence of originally mislabeled test examples increases by just 5%. Test set errors across the 10 datasets can be viewed at https://labelerrors.com and all label errors can be reproduced by https://github.com/cleanlab/label-errors.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/a4f9e7e695bba1ffb90b30752a40d5ee907dcb36

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus (2021, methodology): “[27] found pervasive label errors in test sets from popular benchmarks used in computer vision (e.”
Data and its (dis)contents: A survey of dataset development and use in machine learning research (2020, methodology): “Northcutt et al.(77) combine an algorithmic approach with human validation to surface labeling errors in the test set for ImageNet.”
Distilling Model Failures as Directions in Latent Space (2022, neutral): “For example, a model might struggle on speciﬁc subpopulations in the data if that subpopulation was mislabeled (Northcutt et al., 2021; Stock & Cisse, 2018; Beyer et al., 2020; Vasudevan et al., 2022), underrepresented (Sagawa et al., 2020; Santurkar et al., 2021), or corrupted (Hendrycks &…”
The Benchmark Lottery (2021, background): “Northcutt et al. [2021] identiﬁed label errors in test sets of 10 of the most commonly-used computer vision, natural language, and audio datasets; for example, there are label errors in 6% of the examples in the ImageNet validation set.”
Towards a Standard for Identifying and Managing Bias in Artificial Intelligence (2022, background): “A recent study highlighted serious errors in commonly used benchmark dataset [142].”
Bugs in the Data: How ImageNet Misrepresents Biodiversity (2022, neutral): “There have been numerous studies of ImageNet from different angles, ranging from the quality of its labels (Beyer et al. 2020; Tsipras et al. 2020; Northcutt, Athalye, and Mueller 2021) to the biases that it reflects (Prabhu and Birhane 2021; Crawford and Paglen 2019; Shankar et al. 2017; De Vries…”
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks (2021, background): “ML studies have 79 mainly explored the former through “benchmark misalignment” which arises with systematically 80 unstable labels [22, 23], but misalignment in the context of socio-technical systems is discussed.”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, background): “However, this focus has started to shift in recent years, during which a flurry of ML scholarship has unearthed many issues with even the most commonplace datasets, showing them to be riddled with errors, perpetuating bias, and intensifying discrimination [30, 42, 57, 68, 75].”
Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset (2021, result): “This error rate is consistent with the 3.4% average error rate in the most commonly used test datasets for computer vision, natural language processing, and audio processing [39].”
The Eﬀects of Data Quality on Machine Learning Performance (2022, background): “Northcutt et al. [52] focus on label noise in test data and its eﬀect on ML-benchmark results and thus on model selection.”
Partial success in closing the gap between human and machine vision (2021, background): “H2: because CLIP receives higher-quality labels? About 6% of ImageNet labels are plainly wrong [74].”
Towards Unbounded Machine Unlearning (2023, background): “…where privacy with respect to a subset of individual training examples isn’t necessarily a consideration: removing out-of-date examples, outliers, poisoned samples [Jagielski et al., 2018], noisy labels [Northcutt et al., 2021], or data that may carry harmful biases [Fabbrizzi et al., 2022].”
Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (2021, background): “Several researchers have long articulated concern for the correctness of data labels as an indicator of internal validity [17, 105].”
Advances, challenges and opportunities in creating data for trustworthy AI (2022, methodology): “By looking at data points whose human annotation systematically deviates from predictions made by AI models, the Confidence Learning algorithm identified over 3% of the test data in common benchmarks such as ImageNet that are misannotated 53,54 .”
Confident Learning: Estimating Uncertainty in Dataset Labels (2019, methodology): “, 2020); evaluation of CL methods using other non-neural network models, such as random forests and XGBoost; examination of other threshold function formulations; examination of label errors in test sets and they affect machine learning benchmarks at scale (Northcutt et al., 2021); assimilation of C”