Quantifying Uncertainty in Random Forests via Confidence Intervals and Hypothesis Tests

By Lucas K. Mentch, Giles Hooker. 2014

339 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

Interpreting Models via Single Tree Approximation (2016, background): “Alternatively, where distributional results are available for predictions from the original learner as in Mentch and Hooker [2016], a hypothesis tests of the diﬀerence between the original and approximated predictions could also be employed.”
All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously (2018, methodology): “Mentch and Hooker (2016) use U-statistics to study predictions of ensemble models fit to subsamples, similar to the bootstrap aggregation used in RFs. Procedures related to “Mean Difference Impurity,” another VI measure derived for RFs, have been studied theoretically by Louppe et al. (2013);…”
Local Linear Forests (2018, background): “Mentch and Hooker (2016) then established asymptotic normality of random forests where each tree depends on a small subsample of training examples (so that there may be asymptotic bias), while Wager and Athey (2018) provided a characterization of forests that allows for larger subsamples, deriving…”
Generalized random forests (2016, methodology): “…rather seek an adaptive nearest neighbor weighting that is as sensitive as possible to heterogeneity in our parameter of interest; we then rely on the random forest resampling mechanism to achieve statistical stability (Mentch and Hooker, 2016; Scornet, Biau and Vert, 2015; Wager and Athey, 2018).”
Machine Unlearning for Random Forests (2020, result): “There is also empirical evidence that bootstrapping does not improve predictive performance (Zaman & Hirose, 2009; Denil et al., 2014; Mentch & Hooker, 2016), which was consistent with our own experiments (Appendix: §B.2, Table 5).”
DART: Data Addition and Removal Trees (2020, neutral): “There is empirical evidence that bootstrapping does not improve predictive performance (De-nil, Matheson, and De Freitas 2014; Mentch and Hooker 2016; Zaman and Hirose 2009), and indeed our own pre-liminary experiments did not demonstrate any advantage of a tuned bootstrapped model over a tuned…”
Interpretable Random Forests via Rule Extraction (2020, background): “As it is often the case in the theoretical analysis of random forests, e.g. Scornet et al. (2015); Mentch & Hooker (2016), we assume throughout this section that the subsampling of a n observations prior to each tree construction is done without replacement to alleviate the mathematical analysis.”
Random forest as a generic framework for predictive modeling of spatial and spatio-temporal variables (2018, methodology): “The most202 important parameters are (Probst and Boulesteix, 2017):203 • mtry — number of variables to possibly split at in each node.204 • min.node.size — minimal terminal node size.205 • sample.fraction — fraction of observations to sample in each tree.206 • num.trees — number of trees.207 The num”
Bootstrapping and sample splitting for high-dimensional, assumption-lean inference (2016, background): “Although they are not directed at linear models, [73] and [50] address similar problems for random forests.”
SIRUS: making random forests interpretable (2019, background): “Following Mentch and Hooker (2016), since p n ( P ) is a bagged estimate it can be seen as an inﬁnite-order U-statistic, and a classic bound on the variance of U-statistics gives that V [ p n ( P )] converges to 0 if lim n →∞ a n n = 0 , which is true by Assumption (A1).”
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence (2019, background): “, observations drawn randomly without replacement—which are more theoretically tractable (Mentch and Hooker 2016; Scornet et al. 2015; Wager and Athey 2018).”
Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, methodology): “In particular, the consistency of methods related to Random Forests (RFs: Breiman, 2001) has been demonstrated in Biau (2012); Scornet et al. (2015) while Wager et al. (2014); Mentch and Hooker (2016); Wager and Athey (2017) and Athey et al. (2016) prove central limit theorems for RF predictions.”
Please Stop Permuting Features: An Explanation and Alternatives (2019, methodology): “Note that these methods are distinct from the permute and re-learn approach to testing feature importance in Mentch and Hooker (2016, 2017) as well as the leave-out-covariates (LOCO) approach in Lei et al. (2018).”
On the asymptotics of random forests (2014, result): “However, surprisingly, no study aims at clarifying the link between finite forests (finite M) and infinite forests (M = ∞) even if some authors (Mentch and Hooker, 2014; Wager et al., 2014) proved results on finite forest predictions at a fixed point x.”
Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance (2020, methodology): “Recently, however, Mentch and Hooker (2016) proposed a formal hypothesis testing procedure for measuring feature importance in random forests.”