Variable selection using random forests

By Robin Genuer, Jean‐Michel Poggi, Christine Tuleau-Malot. 2010

1,812 citations recorded by the source corpus.

This paper proposes, focusing on random forests, the increasingly used statistical method for classification and regression problems introduced by Leo Breiman in 2001, to investigate two classical issues of variable selection. The first one is to find important variables for interpretation and the second one is more restrictive and try to design a good parsimonious prediction model. The main contribution is twofold: to provide some experimental insights about the behavior of the variable importance index based on random forests and to propose a strategy involving a ranking of explanatory variables using the random forests score of importance and a stepwise ascending variable introduction strategy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, background): “In addition to outcome prediction, RF ranks variables on the basis of their predictive importance [20].”
Incorporating association rule networks in feature category-weighted naive Bayes model to support weaning decision making (2017, methodology): “forest algorithm [18], and feature selection using decision trees [6].”
Variable selection using Mean Decrease Accuracy and Mean Decrease Gini based on Random Forest (2016, methodology): “It is used to estimate the prediction error and then to evaluate variable importance [4].”
Hyperparameters and tuning strategies for random forest (2018, background): “More trees are generally required for stable variable importance estimates (Genuer et al., 2010; Goldstein et al., 2011), than for the simple prediction purpose.”
Comparison of Variable Selection Methods for Clinical Predictive Modeling (2018, methodology): “The method selects two variable subsets: one used for interpretation that includes all variables highly correlated with the outcome, and one more limited that only includes the smallest subset of variables that are appropriate for prediction [20].”
A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources (2019, methodology): “[82] use a stepwise variable introduction strategy based on ascending VIMs; see Ziegler and König [17] for an assessment of the two approaches.”
A tutorial review: Metabolomics and partial least squares-discriminant analysis--a marriage of convenience or a shotgun wedding. (2015, methodology): “As these methods have been comprehensively described elsewhere [27,30–41], here we shall avoid in-depth and complex descriptions of the mathematics behind these classification models and instead provide simple explanations which include both the advantages and caveats as well as practical applicatio”