Identifying Statistical Bias in Dataset Replication

By Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Jacob Steinhardt, Aleksander Madry. 2020

55 citations recorded by the source corpus.

Dataset replication is a useful tool for assessing whether improvements in test accuracy on a specific benchmark correspond to improvements in models' ability to generalize reliably. In this work, we present unintuitive yet significant ways in which standard approaches to dataset replication introduce statistical bias, skewing the resulting observations. We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality. We show that after correcting for the identified statistical bias, only an estimated $3.6\% \pm 1.5\%$ of the original $11.7\% \pm 1.0\%$ accuracy drop remains unaccounted for. We conclude with concrete recommendations for recognizing and avoiding bias in dataset replication. Code for our study is publicly available at this http URL .

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, background): “Recent work attributes this drop to statistical bias in the data collection [17].”
Are we done with ImageNet? (2020, background): “Later, [24] demonstrate that this discrepancy is explained by a statistical bias in the data collection process that was unintentionally introduced in the replication.”
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization (2020, background): “Recently, [7] identify statistical biases in ImageNetV2’s construction, and they estimate that re-weighting ImageNetV2 to correct for these biases results in a less substantial 3.”
Natural Adversarial Examples (2019, background): “, 2020 [14] estimate that the accuracy drop from ImageNet to ImageNetV2 is less than 3.”
Unsolved Problems in ML Safety (2021, background): “[50] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, D.”
The statistical advantage of automatic NLG metrics at the system level (2021, methodology): “In the replication of ImageNet, Engstrom et al. (2020) found that dataset bias accounted for classiﬁer performance differences between the original and the replicated dataset, and provide a decomposition for the sources of error.”
Improving robustness against common corruptions by covariate shift adaptation (2020, methodology): “[59] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Jacob Steinhardt, and Aleksander Madry.”
Environment Inference for Invariant Learning (2020, background): “Even new test sets collected from the same data acquisition pipeline induce distribution shifts that signiﬁcantly harm performance (Recht et al., 2019; Engstrom et al., 2020).”
From ImageNet to Image Classification: Contextualizing Progress on Benchmarks (2020, background): “4 Note that our selection frequency estimates for ImageNet labels may be biased (underestimates) (Engstrom et al., 2020) as these specific pairs have already been filtered during dataset creation based on their selection frequency.”