Mondrian Forests for Large-Scale Regression when Uncertainty Matters

By Balaji Lakshminarayanan, Daniel M. Roy, Yee Whye Teh. 2015

60 citations recorded by the source corpus.

Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to large-scale data sets comes at the cost of approximating the uncertainty estimates. We extend Mondrian forests, first proposed by Lakshminarayanan et al. (2014) for classification problems, to the large-scale non-parametric regression setting. Using a novel hierarchical Gaussian prior that dovetails with the Mondrian forest framework, we obtain principled uncertainty estimates, while still retaining the computational advantages of decision forests. Through a combination of illustrative examples, real-world large-scale datasets, and Bayesian optimization benchmarks, we demonstrate that Mondrian forests outperform approximate GPs on large-scale regression tasks and deliver better-calibrated uncertainty assessments than decision-forest-based methods.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors (2018, methodology): “We consider the flight delay data set (Hensman et al., 2013; Deisenroth and Ng, 2015; Lakshminarayanan et al., 2016), a large scale regression benchmark with several published results.”
Noise Contrastive Priors for Functional Uncertainty (2018, result): “We consider the flight delay data set (Hensman et al., 2013; Deisenroth and Ng, 2015; Lakshminarayanan et al., 2016), a large scale regression benchmark with several published results.”
NGBoost: Natural Gradient Boosting for Probabilistic Prediction (2019, methodology): “tinguished as Bayesian or non-Bayesian. Bayesian approaches (which include a prior and perform posterior inference) that leverage decision trees for tabular datasets include Chipman et al. (2010) and Lakshminarayanan et al. (2016). A non-Bayesian approach similar to our work is Lakshminarayanan et a”
Bayesian optimization for conditional hyperparameter spaces (2017, background): “What was observed empirically in [14] is that random forests are overconfident in their outputs.”
Hybrid Models with Deep and Invertible Features (2019, unknown): “91 (Lakshminarayanan et al., 2016).”
Bayesian Optimization for Categorical and Category-Specific Continuous Inputs (2019, unknown): “However, random forest has a well-known limitation in performing extrapolation and thus is not a good choice for BO (Lakshminarayanan, Roy, and Teh 2016).”