A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis

By Alexander R. Statnikov, Constantin F. Aliferis, Ioannis Tsamardinos, D. Hardin, S. Levy. Bioinform., 2004

767 citations recorded by the source corpus.

MOTIVATION Cancer diagnosis is one of the most important emerging clinical applications of gene expression microarray technology. We are seeking to develop a computer system for powerful and reliable cancer diagnostic model creation based on microarray data. To keep a realistic perspective on clinical applications we focus on multicategory diagnosis. To equip the system with the optimum combination of classifier, gene selection and cross-validation methods, we performed a systematic and comprehensive evaluation of several major algorithms for multicategory classification, several gene selection methods, multiple ensemble classifier methods and two cross-validation designs using 11 datasets spanning 74 diagnostic categories and 41 cancer types and 12 normal tissue types. RESULTS Multicategory support vector machines (MC-SVMs) are the most effective classifiers in performing accurate cancer diagnosis from gene expression data. The MC-SVM techniques by Crammer and Singer, Weston and Watkins and one-versus-rest were found to be the best methods in this domain. MC-SVMs outperform other popular machine learning algorithms, such as k-nearest neighbors, backpropagation and probabilistic neural networks, often to a remarkable degree. Gene selection techniques can significantly improve the classification performance of both MC-SVMs and other non-SVM learning algorithms. Ensemble classifiers do not generally improve performance of the best non-ensemble models. These results guided the construction of a software system GEMS (Gene Expression Model Selector) that automates high-quality model construction and enforces sound optimization and performance estimation procedures. This is the first such system to be informed by a rigorous comparative analysis of the available algorithms and datasets. AVAILABILITY The software system GEMS is available for download from http://www.gems-system.org for non-commercial use. CONTACT alexander.statnikov@vanderbilt.edu.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/a8244689434f4dcedb7709e4441f5f7433846ce8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Algorithms for Discovery of Multiple Markov Boundaries (2013, methodology): “While a few methods had comparable or slightly higher AUC (Table 3), their Markov boundaries or variable sets were substantially larger with the average PV reaching as high as 41% (see Resampling+UAF in Table 3).”
GEMS: A system for automated cancer diagnosis and biomarker discovery from microarray gene expression data (2005, methodology): “Details on the application of the methods are provided in [3] [4] and references therein.”
Markov blanket-embedded genetic algorithm for gene selection (2007, methodology): “5, Naive Bayes, etc) on twoclass [9] or multi-class [10,21] microarray classification problems.”
Clustering cancer gene expression data: a comparative study (2008, background): “Recently, studies proposing benchmarking frameworks have been introduced in the machine learning and bioinformatics literature [44,45].”
Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation (2010, methodology): “…been applied for the discovery of biomarkers in human cancer data using microarrays and mass spectrometry and is also implemented in the GEMS and FAST-AIMS systems for the automated analysis of microarray and mass spectrometry data respectively (Statnikov et al., 2005b; Fananapazir etal., 2005).”
Machine learning for detection and diagnosis of disease. (2006, unknown): “A recent study has evaluated various types of classifiers for cancer diagnostics, including SVMs, for classification accuracy using a wide array of gene expression microarray data (83).”
A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification (2008, methodology): “The "one-versus-rest" SVM works better for multi-class microarray data [1,6], so we adopted this method for the analysis of multicategory datasets in the present study.”
Machine Learning Meta-analysis of Large Metagenomic Datasets: Tools and Biological Insights (2016, methodology): “The general objectives of meta-analysis include proposing new classifiers [10], comparing different classification methods [11], finding a common transcriptional profile [12], and evaluating generalization of prediction models across different studies [13].”
A review of feature selection techniques in bioinformatics (2007, unknown): “The prevalence of these univariate techniques has dominated the field, and up to now comparative evaluations of different classification and FS techniques over DNA microarray datasets only focused on the univariate case [29, 64, 72, 113 ].”
A comprehensive evaluation of multicategory classification methods for microbiomic data (2013, methodology): “The present work extends prior research by: (i) addressing diagnostic/personalized medicine applications in addition to classification of body sites and subjects, (ii) evaluating a large number of machine learning classification and feature/OTU selection methods, (iii) using more powerful multicateg”
Classification of Gene Expression Data by Majority Voting Genetic Programming Classifier (2006, background): “A comparative study on multicategory classification methods for gene expression data can be found in [21].”
Training highly multiclass classifiers (2014, background): “Statnikov et al. (2005) compared eight multiclass classifiers including that of Weston and Watkins (1999) and Crammer and Singer (2001) on nine cancer classification problems with 3 to 26 classes and less than 400 samples per problem.”
Extension of the Visualization Tool MapMan to Allow Statistical Analysis of Arrays, Display of Coresponding Genes, and Comparison with Known Responses1 (2005, methodology): “Comparison of the global response across many arrays is best done using classiﬁcation approaches (see e.g. Statnikov et al., 2005).”