Bayesian and Empirical Bayesian Forests

By Matt Taddy, Chun-Sheng Chen, Jun Yu, Mitch Wyle. 2015

35 citations recorded by the source corpus.

We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view random forests as samples from a posterior distribution. This insight provides large gains in interpretability, and motivates a class of Bayesian forest (BF) algorithms that yield small but reliable performance gains. Based on the BF framework, we are able to show that high-level tree hierarchy is stable in large samples. This leads to an empirical Bayesian forest (EBF) algorithm for building approximate BFs on massive distributed datasets and we show that EBFs outperform subsampling based alternatives by a large margin.

Subfields: Artificial Intelligence, Information Systems, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Statistics and Probability

Citations in the corpus

Local Linear Forests (2018, methodology): “This model has become a popular study for evaluating nonparametric regression methods (see, e.g., Chipman, George, and McCulloch 2010; Taddy et al. 2015).”
A Nonparametric Bayesian Analysis of Heterogenous Treatment Effects in Digital Experimentation (2014, methodology): “This uncertainty contradicts the examples and results in Taddy et al. (2015), which ﬁnds high posterior probability for the trunks of CART ﬁts and takes advantage of this stability for efﬁcient computation.”
Bitcoin Pricing, Adoption, and Usage: Theory and Evidence (2016, unknown): “More precisely, we interpret the probabilities assigned by the random forest as posterior probabilities from an empirical Bayesian model (for more discussion of this in the context of random forests, see Taddy et al. [2015]).”