Comparison of Variable Selection Methods for Clinical Predictive Modeling

By M. L. N. L. Nelson Sanchez-Pinto, MS Laura Ruth Venable, PhD John Fahrenbach, M. M. P. Matthew M. Churpek. 2018

245 citations recorded by the source corpus.

Objective Modern machine learning-based modeling methods are increasingly applied to clinical problems. One such application is in variable selection methods for predictive modeling. However, there is limited research comparing the performance of classic and modern for variable selection in clinical datasets. Materials and Methods We analyzed the performance of eight different variable selection methods: four regression-based methods (stepwise backward selection using p-value and AIC, Least Absolute Shrinkage and Selection Operator, and Elastic Net) and four tree-based methods (Variable Selection Using Random Forest, Regularized Random Forests, Boruta, and Gradient Boosted Feature Selection). We used two clinical datasets of different sizes, a multicenter adult clinical deterioration cohort and a single center pediatric acute kidney injury cohort. Method evaluation included measures of parsimony, variable importance, and discrimination. Results In the large, multicenter dataset, the modern tree-based Variable Selection Using Random Forest and the Gradient Boosted Feature Selection methods achieved the best parsimony. In the smaller, single-center dataset, the classic regression-based stepwise backward selection using p-value and AIC methods achieved the best parsimony. In both datasets, variable selection tended to decrease the accuracy of the random forest models and increase the accuracy of logistic regression models. Conclusions The performance of classic regression-based and modern tree-based variable selection methods is associated with the size of the clinical dataset used. Classic regression-based variable selection methods seem to achieve better parsimony in clinical prediction problems in smaller datasets while modern tree-based methods perform better in larger datasets.

Subfields: Artificial Intelligence, Epidemiology, Health Informatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nan Liu
2. Marcus Eng Hock Ong
3. Edward Hak-Sing Ip
4. Jaime Lynn Speiser
5. Janet Tooze
6. Michael I. Miller

In Epidemiology

1. Nan Liu
2. Marcus Eng Hock Ong
3. Feng Xie
4. Benjamin Alan Goldstein
5. Bibhas Chakraborty

Citations in the corpus

• AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, methodology): “Sanchez-Pinto et al [11] also provided evidence of modern tree-based methods of variable selection with better parsimony in large data sets.”
• A Comparison of Random Forest Variable Selection Methods for Classification Prediction Modeling (2019, background): “Consistent with findings from Sanchez-Pinto (2018), our study found that VSURF had slightly better prediction error compared to Boruta and RRF. Degenhardt (2017) focused on omics datasets which were quite large, but concluded that Boruta and Altmann were suitable for low dimensional data; however,…”