Understanding variable importances in forests of randomized trees

By Gilles Louppe, Louis Wehenkel, Antonio Sutera, Pierre Geurts. 2013

1,069 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, methodology): “This approach is refined in Deng and Runger (2013), by also including the importance scores of the predictor variables (Louppe et al. 2013; Strobl et al. 2008), obtained in a preprocessing step running a preliminary RF.”
AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, background): “The importance measurement of a variable Xm is the weighted total of impurity decreases w( )ΔGini( ) for all nodes , averaged over all trees [21]:”
All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously (2018, methodology): “…use U-statistics to study predictions of ensemble models fit to subsamples, similar to the bootstrap aggregation used in RFs. Procedures related to “Mean Difference Impurity,” another VI measure derived for RFs, have been studied theoretically by Louppe et al. (2013); Kazemitabar et al. (2017).”
A random forest guided tour (2015, background): “Louppe et al. (2013) define a variable X ( j) as irrelevant with respect to B ⊂ V = X1 × · · · × Xp whenever I (X ( j); Y |B) = 0.”
VSURF: An R Package for Variable Selection Using Random Forests (2015, result): “A similar result for the mean decrease impurity index (not used in this paper) has been proven by Louppe et al. (2013).”
All Models are Wrong but many are Useful: Variable Importance for Black-Box, Proprietary, or Misspecified Prediction Models, using Model Class Reliance (2018, methodology): “Procedures related to “Mean Difference Impurity,” another VI measure derived for RFs, have been studied theoretically by (Louppe et al., 2013; Kazemitabar et al., 2017).”
iForest: Interpreting Random Forests via Visual Analytics (2019, methodology): “, Gini impurity [24]), which determines the feature and its split point for each node in a decision tree [37].”
Variable Importance Using Decision Trees (2017, methodology): “The closest related work is the analysis of [8], which considers a fixed set of variables, in the limit of infinite data (n =∞).”
A Survey of Android Malware Detection with Deep Neural Models (2020, methodology): “To reduce feature dimension and improve efficiency, mean decrease impurity in Random Forests [111] was used to select the most significant features.”
Machine Learning Predicts Laboratory Earthquakes (2017, methodology): “We apply a machine learning technique, the random forest (RF) (Breiman, 2001; Louppe et al., 2013; Pedregosa et al., 2011), to the continuous acoustic time series data recorded from the fault (see Figure 1 and Texts S2 and S3).”
Fair Forests: Regularized Tree Induction to Minimize Model Bias (2017, methodology): “Using the Mean Decrease in Impurity (MDI) measure [21], we show the relative importance of features on the German and Adult datasets.”
Secrets of GrabCut and Kernel K-Means (2015, background): “Both Gini and entropy clustering criteria are widely used for decision trees [7, 22].”
Learning Decision Trees Recurrently Through Communication (2021, methodology): “For aDT we use the Mean Decrease Impurity (MDI) [39] to find features of maximum importance and for aResNet we select the attributes with the highest weight for the output class.”
Explaining Explanations to Society (2019, methodology): “In this paper, we explicitly examine DNNs, even though there have been important developments in explanatory systems not tailored to DNNs, such as randomized importance [20], rule lists [19], partial dependence plots [9, 22, 37], Shapely scores [22, 31], and Bayesian case based models [15].”
Understanding Random Forests: From Theory to Practice (2014, methodology): “They may not tell the entire nor the same story! (Louppe et al., 2013)”
MEMOTE for standardized genome-scale metabolic model testing (2020, methodology): “, test case, was ranked with the Mean Decrease in Accuracy (MDA) (Louppe et al. 2013).”