Neural Random Forests

By Gérard Biau, Erwan Scornet, Johannes Welbl. 2016

122 citations recorded by the source corpus.

Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights. Following this principle, we reformulate the random forest method of Breiman (2001) into a neural network setting, and in turn propose two new hybrid procedures that we call neural random forests. Both predictors exploit prior knowledge of regression trees for their architecture, have less parameters to tune than standard networks, and less restrictions on the geometry of the decision boundaries than trees. Consistency results are proved, and substantial numerical evidence is provided on both synthetic and real data sets to assess the excellent performance of our methods in a large variety of prediction problems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Black-Box vs. White-Box: Understanding Their Advantages and Weaknesses From a Practical Point of View (2019, background): “tory models to provide an explanation close to the human language [151], [152], [156].”
Machine Unlearning for Random Forests (2020, methodology): “, 2018), mainly due to their predictive prowess on many classification and regression tasks (Kocev et al., 2013; Genuer et al., 2017; Wager & Athey, 2018; Linero & Yang, 2018; Biau et al., 2019).”
Neural Decision Trees (2017, methodology): “Finally and more recently, a reformulation of regression tree as a sparse ANN as been done in Biau et al. (2016) in order to ﬁne-tune the learned DT.”
DART: Data Addition and Removal Trees (2020, unknown): “…2001) are popular and widely used machine learning models (Lundberg, Erion, and Lee 2018), mainly due to their predictive prowess on many classiﬁcation and regression tasks (Biau, Scornet, and Welbl 2019; Kocev, Vens et al. 2013; Genuer et al. 2017; Wager and Athey 2018; Linero and Yang 2018).”
The Tree Ensemble Layer: Differentiability meets Conditional Computation (2020, methodology): “Biau et al. (2019) represented a decision tree using a 3-layer neural network and combined CART and SGD for training.”
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation (2020, result): “While neural nets trained in the standard fashion are usually less accurate than trees for tabular data, FAST-DAD can boost their performance above that of trees, a goal other research has struggled to reach [45–49].”