Model Similarity Mitigates Test Set Overuse

By Horia Mania, John Miller, Ludwig Schmidt, Moritz Hardt, B. Recht. 2019

61 citations recorded by the source corpus.

Excessive reuse of test data has become commonplace in today's machine learning workflows. Popular benchmarks, competitions, industrial scale tuning, among other applications, all involve test data reuse beyond guidance by statistical confidence bounds. Nonetheless, recent replication studies give evidence that popular benchmarks continue to support progress despite years of extensive reuse. We proffer a new explanation for the apparent longevity of test data: Many proposed models are similar in their predictions and we prove that this similarity mitigates overfitting. Specifically, we show empirically that models proposed for the ImageNet ILSVRC benchmark agree in their predictions well beyond what we can conclude from their accuracy levels alone. Likewise, models created by large scale hyperparameter search enjoy high levels of similarity. Motivated by these empirical observations, we give a non-asymptotic generalization bound that takes similarity into account, leading to meaningful confidence bounds in practical settings.

Subfields: Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

The Effect of Natural Distribution Shift on Question Answering Models (2020, background): “A number of works have proffered explanations for why adaptive overﬁtting does not occur in machine learning (Blum & Hardt, 2015; Mania et al., 2019; Feldman et al., 2019; Zrnic & Hardt, 2019).”
Datamodels: Predicting Predictions from Training Data (2022, background): “as datamodels are fit to a particular learning algorithm; at the same time, we also expect some transfer of predictive power as modern deep neural networks are known to make similar predictions and errors (Mania et al., 2019).”
Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (2021, background): “An active line of research investigates the question of overﬁtting from test set reuse, also known as adaptive overﬁtting [5, 9, 14, 37, 42, 91, 175].”
The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning (2021, background): “Empirically, [22] observed that the labeling assignments across a wide range of ImageNet models are signiﬁcantly more similar to each other than would be expected by chance, which suggests that the size of the class of function approximators that neural networks learn is smaller than what may be…”
Partial success in closing the gap between human and machine vision (2021, background): “Irrespective of whether one takes a standard supervised model, a selfsupervised model, an adversarially trained model or a vision transformer, all those models make highly systematic errors (which extends the results of [32, 76] who found similarities between standard vanilla CNNs).”
No One Representation to Rule Them All: Overlapping Features of Training Methods (2021, background): “Mania et al. (2019) demonstrates, via “dominance probabilities”, that high-accuracy models rarely make mistakes that low-accuracy models predict correctly.”
Forward Compatible Training for Large-Scale Embedding Retrieval Systems (2021, unknown): “It is well-known in deep learning literature that different runs of SGD for the same model produce diverse predictions [14, 29], which is particularly useful for ensembling.”
Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency (2020, background): “Closely related to our analysis is [27], who investigated similarity between models in the context of overfitting.”
A Meta-Analysis of Overfitting in Machine Learning (2019, background): “The adaptive data analysis literature [6, 17] provides a range of theoretical explanations for how the common machine learning workﬂow may implicitly mitigate overﬁtting [3, 8, 12, 23].”