Grouped variable importance with random forests and application to multiple functional data analysis

By Baptiste Gregorutti, Bertrand Michel, Philippe Saint-Pierre. 2014

140 citations recorded by the source corpus.

The selection of grouped variables using the random forest algorithm is considered. First a new importance measure adapted for groups of variables is proposed. Theoretical insights into this criterion are given for additive regression models. Second, an original method for selecting functional variables based on the grouped variable importance measure is developed. Using a wavelet basis, it is proposed to regroup all of the wavelet coefficients for a given functional variable and use a wrapper selection algorithm with these groups. Various other groupings which take advantage of the frequency and time localization of the wavelet basis are proposed. An extensive simulation study is performed to illustrate the use of the grouped importance measure in this context. The method is applied to a real life problem coming from aviation safety.

Subfields: Computational Theory and Mathematics, Environmental Engineering, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Environmental Engineering

In Statistics and Probability

Citations in the corpus

All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously (2018, methodology): “…are provided at http://jmlr.org/papers/v20/18-760.html. the decrease in prediction accuracy when a covariate is permuted (Breiman, 2001; Breiman et al., 2001; see also Strobl et al., 2008; Altmann et al., 2010; Zhu et al., 2015; Gregorutti et al., 2015; Datta et al., 2016; Gregorutti et al., 2017).”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies (2021, background): “followed by a rich line of research on the topic [4, 34, 56, 57, 131, 156].”
Please Stop Permuting Features: An Explanation and Alternatives (2019, methodology): “Hooker (2007) provided an interpretation of all these measures in the context of the functional ANOVA decomposition (Hoeffding, 1948) but only when every feature is statistically independent from all the rest. A generalization can be obtained when the features are dependent, but this does not readil”
Multivariate analysis of variance for functional data (2017, methodology): “Particular problems considered in functional data analysis include canonical correlation analysis [29,30,33], cluster analysis [27,49], confidence intervals [32], discriminant analysis [4,18,47], linear and nonlinear models [3,15,35], the one-way ANOVAproblem [12,14,20,25,26,50,51], principal compon”
All Models are Wrong but many are Useful: Variable Importance for Black-Box, Proprietary, or Misspecified Prediction Models, using Model Class Reliance (2018, methodology): “Outside of the context of RF VI, Zhu et al. (2015) propose an estimand similar to our definition of model reliance, and (Gregorutti et al., 2015, 2017) propose an estimand analogous to eswitch(f) − eorig(f).”