BART: Bayesian Additive Regression Trees

By Hugh A. Chipman, Edward I. George, Robert E. McCulloch. 2008

2,052 citations recorded by the source corpus.

We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Subfields: Artificial Intelligence, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation (2013, methodology): “…thanks to PDPs. Green and Kern (2010) used PDPs to understand the relationship between predictors and the conditional average treatment effect for a voter mobilization experiment, with the predictions being made by Bayesian additive regression trees (BART, Chipman, George, and McCulloch 2010).”
Fair Inference on Outcomes (2017, methodology): “As an example, we consider BART, a popular Bayesian random forest method described in (Chipman, George, and McCulloch 2010).”
Learning Certifiably Optimal Rule Lists (2017, methodology): “Some exploration of the search space is done by Bayesian decision tree methods [7, 8, 12] and Bayesian rule-based methods [25, 45].”
All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously (2018, methodology): “…model classes where profile likelihoods are difficult to compute, such as neural networks or random forests, approximate inference is sometimes achieved via approximate optimization procedures (for example, Markov chain Monte Carlo for Bayesian additive regression trees, in Chipman et al., 2010).”
FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference (2017, methodology): “, 2015), (2) 1-PSNNM with oracle variable selection, (3) Genetic Matching (GenMatch) (Diamond and Sekhon, 2013), (4) Causal Forest (Wager and Athey, 2018), (5) Mahalanobis Matching, (6) double linear regression, (7) BART (Chipman et al., 2010) and (8) CTMLE (Van Der Laan and Rubin, 2006).”
Local Linear Forests (2018, methodology): “We compare local linear forests with ordinary least squares (OLS), lasso with interaction terms, gradient boosting, Bayesian additive regression trees (BART), and random forest.”
Metalearners for estimating heterogeneous treatment effects using machine learning (2017, methodology): “For BART, our software used the dbarts (18) implementation for the base learner.”
Generalized random forests (2016, methodology): “Our asymptotic theory relates to an extensive recent literature on the statistics of random forests, most of which focuses on the regression case (Arlot and Genuer, 2014; Biau, 2012; Biau et al., 2008; Biau and Scornet, 2016; Breiman, 2004; Bühlmann and Yu, 2002; Chipman et al., 2010; Denil et al., ”
The Econometrics of Randomized Experiments (2016, methodology): “These papers use the Bayesian additive regression tree method of Chipman et al (2010), and report posterior credible intervals obtained by Markov-chain Monte Carlo sampling based on a convenience prior.”
The State of Applied Econometrics - Causality and Policy Evaluation (2016, neutral): “See for example the discussion in Deaton [2010], Imbens [2010] and Manski [2013]. Traditionally, there has been much emphasis on internal validity in studies of causal effects, with some arguing for the primacy of internal validity.”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, background): “Notable extensions of the original random forest algorithm include online random forests [15, 34], random survival forests [23] and Bayesian alternatives to random forests [13].”
Heterogeneous Treatment Effects in Digital Experimentation (2014, methodology): “For example, Hill (2011) applies Bayesian additive regression trees (BART; Chipman et al., 2010) to obtain posterior distributions for tree-based regression in each treatment group.”