A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification

By Alexander R. Statnikov, Lily Wang, Constantin F. Aliferis. BMC Bioinformatics, 2008

690 citations recorded by the source corpus.

BackgroundCancer diagnosis and clinical outcome prediction are among the most important emerging applications of gene expression microarray technology with several molecular signatures on their way toward clinical deployment. Use of the most accurate classification algorithms available for microarray gene expression data is a critical ingredient in order to develop the best possible molecular signatures for patient care. As suggested by a large body of literature to date, support vector machines can be considered "best of class" algorithms for classification of such data. Recent work, however, suggests that random forest classifiers may outperform support vector machines in this domain.ResultsIn the present paper we identify methodological biases of prior work comparing random forests and support vector machines and conduct a new rigorous evaluation of the two algorithms that corrects these limitations. Our experiments use 22 diagnostic and prognostic datasets and show that support vector machines outperform random forests, often by a large margin. Our data also underlines the importance of sound research design in benchmarking and comparison of bioinformatics algorithms.ConclusionWe found that both on average and in the majority of microarray datasets, random forests are outperformed by support vector machines both in the settings when no gene selection is performed and when several popular gene selection methods are used.

Subfields: Cancer Research, Molecular Biology, Periodontics

View Paper: https://www.semanticscholar.org/paper/dbdc419e0f20e53d4ad3ca18ad808f50e8d20724

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cancer Research

In Molecular Biology

In Periodontics

Citations in the corpus

Testing the reliability and stability of the internal accuracy assessment of random forest for classifying tree defoliation levels using different validation methods (2015, result): “(2007) have supported this claim, others such as Bylander (2002), Ruiz-Gazen and Villa (2007), (Statnikov et al. 2008), Strobl and Zeileis (2008) and Menze et al.”
A Review of Ensemble Methods in Bioinformatics (2010, methodology): “[46] compared random forests with SVM for microarray-based cancer classification across 22 datasets.”
Data mining for credit card fraud: A comparative study (2011, background): “Support vector machines and random forests are sophisticated data mining techniques which have been noted in recent years to show superior performance across different applications [30] [38] [46] [49].”
Design of the 2015 ChaLearn AutoML challenge (2015, background): “Although several authors have increased awareness of good practice of numerical experimental design to reduce the risk of overfitting [41], [42], in particular by splitting data in a principled way [53], to our knowledge, no one has addressed the problem of optimally splitting data.”
Application of machine learning methods to spatial interpolation of environmental variables (2011, result): “SVM was found to be superior to RF in a previous study (Statnikov et al., 2008), but our results showed that the machine learning methods like SVM and KSVM performed more poorly than RF and were even less accurate than IDW in this study.”
Analysis of the AutoML Challenge Series 2015-2018 (2019, background): “Although progress has been made in experimental design to reduce the risk of overfitting (Ioannidis, 2005; Langford, 2005), in particular by splitting data in a principled way (Statnikov et al., 2008), to our knowledge, no one has addressed the problem of optimally splitting data.”
Decision Forests for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning (2011, background): “the machine learning, vision and medical imaging literature [9, 15, 24, 29, 33, 35, 52, 53, 54, 56, 58, 59, 60, 61, 65, 70, 80, 83, 88, 102].”
A tutorial review: Metabolomics and partial least squares-discriminant analysis--a marriage of convenience or a shotgun wedding. (2015, background): “However, random forests can tend to overfit under some data distributions where maximal unpruned trees are used, consequently reducing performance of the model which is something that needs to be considered when applying this approach [137].”
Predictive soil parent material mapping at a regional-scale: a Random Forest approach. (2014, background): “RF haswidely beenused in thefield of bioinformatics (ie. Díaz-Uriarte and Alvarez de Andrés, 2006; Qi, 2012; Statnikov et al., 2008; Svetnik et al., 2003, 2004).”
Random Forests for dependent data (2020, neutral): “performance of RF [65].”
A New Multi-Task Learning Technique to Predict Classification of Leukemia and Prostate Cancer (2010, methodology): “Among these techniques, SVM is touted as having the best accuracy in the prediction of cancer classification [1, 15].”
Mapping Soil Properties of Africa at 250 m Resolution: Random Forests Significantly Improve Current Predictions (2015, background): “[39] have shown that random forests may also over-fit data sets that are particularly noisy.”
Boruta – A System for Feature Selection (2010, methodology): “The relative performance of SVM and RF in various bioinformatical applications is disputed [6, 15], nevertheless the RF method provides two key advantages: it does not need any parameter optimisation to reach its highest performance and it has the built-in statistical estimation of the variable impo”
Support Vector Machine Versus Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review (2020, background): “SVMs [78]; however, no such a detailed survey is available for RS image classification.”
Automated Machine Learning: Methods, Systems, Challenges (2019, neutral): “[61] train an SVM meta-regressor per classification algorithm to predict its accuracy, under default settings, on a new task tnew given its meta-features.”
Novel Speech Signal Processing Algorithms for High-Accuracy Classification of Parkinson's Disease (2012, background): “[40] remark, this undertaking is not straightforward, and requires extensive empirical and theoretical studies to explain the performance differences observed across different studies for SVMs and RF [36].”
Ten quick tips for machine learning in computational biology (2017, background): “Popular supervised learning algorithms in computational biology are support vector machines (SVMs) [19], k -nearest neighbors ( k -NN) [20], and random forests [21].”