Random Forests for Big Data

By Robin Genuer, Jean‐Michel Poggi, Christine Tuleau-Malot, Nathalie Villa-Vialaneix. 2015

357 citations recorded by the source corpus.

Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like linear regression models, clustering methods and bootstrapping schemes. Based on decision trees combined with aggregation and bootstrap ideas, random forests were introduced by Breiman in 2001. They are a powerful nonparametric statistical method allowing to consider in a single and versatile framework regression problems, as well as two-class and multi-class classification problems. Focusing on classification problems, this paper proposes a selective review of available proposals that deal with scaling random forests to Big Data problems. These proposals rely on parallel environments or on online adaptations of random forests. We also describe how related quantities -- such as out-of-bag error and variable importance -- are addressed in these methods. Then, we formulate various remarks for random forests in the Big Data context. Finally, we experiment five variants on two massive datasets (15 and 120 millions of observations), a simulated one as well as real world data. One variant relies on subsampling while three others are related to parallel implementations of random forests and involve either various adaptations of bootstrap to Big Data or to "divide-and-conquer" approaches. The fifth variant relates on online learning of random forests. These numerical experiments lead to highlight the relative performance of the different variants, as well as some of their limitations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Systems

Citations in the corpus

Optimal randomized classification trees (2021, background): “One of the most popular extensions of CARTs is Random Forests (RFs) [34, 35, 36, 37], which involve building a collection of unpruned decision trees.”
DART: Data Addition and Removal Trees (2020, neutral): “…2001) are popular and widely used machine learning models (Lundberg, Erion, and Lee 2018), mainly due to their predictive prowess on many classiﬁcation and regression tasks (Biau, Scornet, and Welbl 2019; Kocev, Vens et al. 2013; Genuer et al. 2017; Wager and Athey 2018; Linero and Yang 2018).”
Sparsity in Optimal Randomized Classification Trees (2020, background): “Standard CARTs or Random Forests (RFs) [5, 7, 13, 16] cannot manage it due to the greedy construction of the trees.”
Machine Unlearning for Random Forests (2020, methodology): “, 2018), mainly due to their predictive prowess on many classification and regression tasks (Kocev et al., 2013; Genuer et al., 2017; Wager & Athey, 2018; Linero & Yang, 2018; Biau et al., 2019).”
A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources (2019, background): “[48]), with all applicable reservations and constraints regarding the blind use of such models in exploring data sets (see e.”