Study on the Impact of Partition-Induced Dataset Shift on $k$-Fold Cross-Validation

By Jose Garcia Moreno-Torres, José A. Sáez, Francisco Herrera. IEEE Transactions on Neural Networks and Learning Systems, 2012

313 citations recorded by the source corpus.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/3f78ea66a8a5d3c652fd176748098be76ea1d12e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Contrast Pattern-Based Classification for Bot Detection on Twitter (2019, methodology): “In our experiments, we have used k = 3 because several authors as [66] and [67] have reported it yields better classification results than other k values.”
Reliable Accuracy Estimates from k-Fold Cross Validation (2020, background): “The number of folds is generally set to be 2, 5, or 10 in exploring the characteristics of k -fold cross validation [2], [13], [14], [18].”
A note on the validity of cross-validation for evaluating autoregressive time series prediction (2018, methodology): “…2010) is one of the most widely used methods to assess the generalizability of algorithms in classification and regression (Hastie et al., 2009), and is subject to ongoing active research (e.g., Budka and Gabrys, 2013; Borra and Di Ciaccio, 2010; Bergmeir et al., 2014; Moreno-Torres et al., 2012).”
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary (2018, background): “In particular, we may find in the work of Moreno-Torres et al. (2012b) a novel approach that is not biased to this problem.”
A review of microarray datasets and applied feature selection methods (2014, methodology): “However, it has been shown [84] that cross-validation can potentially introduce dataset shift (see Section 4.”
A Pattern-Based Approach for Detecting Pneumatic Failures on Temporary Immersion Bioreactors (2019, methodology): “We partitioned all databases using five-fold and distribution optimally balanced stratified cross validation (DOB-SCV) [37] with the goal of avoiding problems about data distribution imbalanced databases [38].”
Cross-Validation for Imbalanced Datasets: Avoiding Overoptimistic and Overfitting Approaches [Research Frontier] (2018, background): “A promising approach to surpass the issues of dataset shift is the Distribution Optimally Balanced stratified cross-validation (DOB-SCV) [46], which is worthy of investigation in future works in the field.”