Consistency of Online Random Forests

By Misha Denil, David Matheson, Nando de Freitas. 2013

93 citations recorded by the source corpus.

As a testament to their success, the theory of random forests has long been outpaced by their application in practice. In this paper, we take a step towards narrowing this gap by providing a consistency result for online random forests.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, background): “The reader is referred to Biau et al. (2008), Denil et al. (2013), Scornet (2016), and Scornet et al. (2015) for some results in this line.”
Uniform Convergence of Random Forests via Adaptive Concentration (2015, methodology): “Proof of Lemma 15.”
On the asymptotics of random forests (2014, methodology): “Consistency of such simplified models has been proved (e.g., Biau et al., 2008; Ishwaran and Kogalur, 2010; Denil et al., 2013).”
Joint Training of Generic CNN-CRF Models with Stochastic Optimization (2015, background): “For this reason, in [35], a set of synthetically rendered depth images along with the corresponding ground truth labelings were introduced (see examples in Fig.”
A random forest guided tour (2015, background): “Random forests have been extended to the online framework in several ways (Saffari et al., 2009; Denil et al., 2013; Lakshminarayanan et al., 2014).”
Ensemble learning for data stream analysis: A survey (2017, methodology): “Bagging-based OzaBag [137] Online Bagging ASHT [17] Ensemble of adaptive-size Hoeffding trees LevBag [16] Leveraging Bagging with increased resampling and output detection codes ORF [41,153] Online Random Forest MF [111] Online Mondrian Forest Boosting-based OzaBoost [137] Online Boosting Others UFF”
Narrowing the Gap: Random Forests In Theory and In Practice (2013, background): “In this way, each tree in the forest is trained on slightly different data, which introduces differences between the trees.”
Online Machine Learning in Big Data Streams (2018, methodology): “ase models are trained based on samples with replacement. Sampling with replacement can be simulated online [181, 28]. A special ensemble technique, the online random forest algorithm is described in [70]. A highly successful ensemble technique is boosting, in which we generate a sequence of base mo”
Understanding Random Forests: From Theory to Practice (2014, methodology): “Finally, let us complete this review by mentioning consistency results in the case of domain-specific adaptations of random forests, including quantile regression [Meinshausen, 2006], survival analysis [Ishwaran and Kogalur, 2010] and online forest construction [Denil et al., 2013b].”
Efficient Likelihood Learning of a Generic CNN-CRF Model for Semantic Segmentation (2015, methodology): “, 2015), which extended (Denil et al., 2013), and is also based on random forests.”
Random Forests for Big Data (2015, background): “Key words: Random Forest, Big Data, Parallel Computing, Bag of Little Bootstraps, On-line Learning”
Mondrian Forests: Efficient Online Random Forests (2014, methodology): “Existing online random forests (ORF-Saffari [20] and ORF-Denil [8]) start with an empty tree and grow the tree incrementally.”