A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

By Ron Kohavi. 1995

Subfields: Statistics and Probability, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Evaluation & Benchmarks

Citations in the corpus

Style transfer-based image synthesis as an efficient regularization technique in deep learning (2019, methodology): “In total, we prepared 5-fold crossvalidation [38] sub-sets.”
Robust Classification for Imprecise Environments (2000, methodology): “Applied statistics provides methods such as cross-validation and the bootstrap for estimating model error rates and recent studies have compared the eﬀectiveness of diﬀerent methods (Dietterich, 1998; Kohavi, 1995; Salzberg, 1997).”
A framework for species distribution modelling with improved pseudo-absence generation (2015, methodology): “In particular, we used a 10-fold cross 140 validation approach, given that it is equally efficient in the error estima141 tion as other techniques computationally more demanding like for instance 142 leave-one-out cross validation (Kohavi, 1995).”
Domain Generalization by Marginal Transfer Learning (2017, methodology): “In order to stabilize the cross-validation procedure, it was repeated 5 times over independent random splits into folds (Kohavi, 1995).”
A review on neuroimaging‐based classification studies and associated feature extraction methods for Alzheimer's disease and its prodromal stages (2017, unknown): “Cross-validation is generally designed to achieve independent training and test data for a classification algorithm and defined as split the data once (split-in-train-test) or several times (kfold cross-validation) to obtain an unbiased estimate of the classification performance of the algorithm and”
Methodology for long-term prediction of time series (2007, methodology): “A 10-fold cross-validation [9] procedure for model structure selection purposes has been applied.”
A Wearable Smartphone-Based Platform for Real-Time Cardiovascular Disease Detection Via Electrocardiogram Processing (2010, methodology): “Furthermore, the whole training process is guided and assessed by the validation set of inputs using cross-validation strategy [15] that is able to help in preventing overfitting.”
Learning from examples to improve code completion systems (2009, unknown): “To achieve a standard machine learning evaluation, we repeat the evaluation process 10 times; this is known as 10-fold cross validation [18].”
Characterising performance of environmental models (2013, background): “is is repeated for all data points, each one in turn being left out (Kohavi, 1995).”
Cross-validation pitfalls when selecting and assessing regression and classification models (2014, methodology): “Kohavi [6] and Hastie et al. [4] empirically show that V-fold cross-validation compared to leave-one-out cross-validation has lower variance, and therefore tends to select simpler models.”
Design of the 2015 ChaLearn AutoML challenge (2015, background): “Recent efforts building on top of old concepts [11]–[17] showcase what is possible to create software that can be used out-of-the-box by ML novices [18]–[20].”
Automated Discovery of Process Models from Event Logs: Review and Benchmark (2017, methodology): “To measure generalization we use 3-fold cross validation [58]: We divide the log into three parts, discover a model from two parts (i.e. we hold-out one part), and measure the ﬁtness of the discovered model against the part held out.”