Is cross-validation valid for small-sample microarray classification?

By Ulisses M. Braga-Neto, Edward R. Dougherty. Bioinform., 2004

638 citations recorded by the source corpus.

Subfields: Analytical Chemistry, Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/f38a69c657bbb7927ef032049c9ea9ba17322b15

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

A review of feature selection techniques in bioinformatics (2007, positive): “Small sample sizes, and their inherent risk of imprecision and overfitting, pose a great challenge for many modelling problems in bioinformatics (Braga-Neto and Dougherty, 2004; Molinaro et al ., 2005; Sima and Dougherty, 2006).”
Kernel Methods and Machine Learning (2014, background): “It has been demonstrated in a microarray application that bootstrap provides the best performance in terms of the variance of the estimate, but at a high computational cost and some increase in bias (although much less than that of re-substitution) [28].”
Prediction error estimation: a comparison of resampling methods (2005, methodology): “In two recent assessments of resampling techniques for error estimation ( Braga-Neto and Dougherty, 2004; Efron, 2004), feature selection was not included as part of the resampling procedures, causing the conclusions to be inapplicable to the high-dimensional setting.”
A Survey on Evolutionary Computation Approaches to Feature Selection (2016, neutral): “Although cross-validation or bootstrap sampling techniques [249] can address the issue to some extent, they may have the problem of it being hard to”
The Impact of Automated Parameter Optimization on Defect Prediction Models (2018, background): “Prior studies have also shown that 10-fold cross validation can produce unstable results for small samples [17].”
Markov blanket-embedded genetic algorithm for gene selection (2007, methodology): “Furthermore, a comparison of various error estimation methods on microarray classiflcation [ 43 ] also suggests that”
Comparison of support vector machine, random forest and neural network classifiers for tree species classification on airborne hyperspectral APEX images (2017, methodology): “Based on the work by Braga-Neto and Dougherty (2014)and Ghosh et al. (2014), we decided to use the 0.632 bootstrap approach for generating the test and training dataset.”
Optimal number of features as a function of sample size for various classification rules (2005, methodology): “…, but such methods are very inaccurate in the sense that the expected absolute deviation between the estimated error and the true error is often very large, the situation being worse for complex classification rules and with increasing numbers of features (Braga-Neto and Dougherty, 2004b).”
Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation (2010, methodology): “Ak-cv error estimator is an unbiased estimator of the prediction error � on data sets of n � n=k size [2], but it is biased for � on data sets of sizen because only a subset of the instances with sizen � n=k isusedfortraining.Thisiscalledthesurrogateproblem[ 5 ].Intuitively, this characteristic will”
A Review of Ensemble Methods in Bioinformatics (2010, background): “This is commonly known as the “curse-of-dimensionality” [26], and selecting the most relevant features [27, 28] and making the most use of the limited data samples [29] are the key issues in microarray or MS-based proteomics classification problem.”
A comprehensive evaluation of multicategory classification methods for microbiomic data (2013, neutral): “, 1) clop [15,27,28] (clopinet.”
Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap (2009, methodology): “There has not been much study on the repeated CV. Braga-Neto and Dougherty (2004) studied the repeated CV and the bootstrap for a small-sample microarray classification, and concluded that the .632 bootstrap estimator is the best in their simulation study with the caution that it might be susceptibl”
Robust biomarker identification for cancer diagnosis with ensemble feature selection methods (2010, methodology): “Since we are considering typically 500 independent partitionings into 90% training and 10% validation, we reduce the risk of over-optimistic results of traditional cross-validation experiments on small sample domains (Braga-Neto and Dougherty, 2004).”
Computational Intelligence and Feature Selection - Rough and Fuzzy Approaches (2008, methodology): “Although such techniques as cross-validation and bootstrapping have been proposed as a way of countering modeling inaccuracies, these still often exhibit high variance in error estimation [42].”
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models (2017, methodology): “The design our experiments of Section 5 take the approaches that have been used in the other research areas into consideration [11], [45], [55], [58], [59].”
Automated Parameter Optimization of Classification Techniques for Defect Prediction Models (2016, methodology): “While this bootstrap resampling approach is common in other research areas [3, 8, 9], recent studies in software engineering tend to estimate the performance of defect models using data from different contexts [63].”
A review of microarray datasets and applied feature selection methods (2014, methodology): “In [29], an extensive simulation study was performed comparing cross-validation, resubstitution and bootstrap estimation.”