To tune or not to tune the number of trees in random forest?

By Philipp Probst, Anne-Laure Boulesteix. 2017

460 citations recorded by the source corpus.

The number of trees T in the random forest (RF) algorithm for supervised learning has to be set by the user. It is controversial whether T should simply be set to the largest computationally manageable value or whether a smaller T may in some cases be better. While the principle underlying bagging is that "more trees are better", in practice the classification error rate sometimes reaches a minimum before increasing again for increasing number of trees. The goal of this paper is four-fold: (i) providing theoretical results showing that the expected error rate may be a non-monotonous function of the number of trees and explaining under which circumstances this happens; (ii) providing theoretical results showing that such non-monotonous patterns cannot be observed for other performance measures such as the Brier score and the logarithmic loss (for classification) and the mean squared error (for regression); (iii) illustrating the extent of the problem through an application to a large number (n = 306) of datasets from the public database OpenML; (iv) finally arguing in favor of setting it to a computationally feasible large number, depending on convergence properties of the desired performance measure.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Automated Machine Learning: State-of-The-Art and Open Challenges (2019, background): “data scientist in each of these steps affect the performance and the quality of the developed model [122, 100, 95].”
AutoScore: A Machine Learning–Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records (2020, methodology): “The parameters were selected according to the suggestions in previous literature [28,29], where ntree=100 and mtry=m (ntree: the number of trees grown; mtry: the number of variables randomly sampled as candidates at each split).”
How to explain and predict the shape parameter of the generalized extreme value distribution of streamflow extremes using a big dataset (2018, background): “The number of trees is set equal to 1 000 in the present study, since the gain in the predictive performance of the algorithm would be small by adding more trees (e.g. Probst and Boulesteix 2018).”
A Brief Review of Random Forests for Water Scientists and Practitioners and Their Recent History in Water Resources (2019, methodology): “Variable importance metrics (VIMs) are not always reliable, as they are affected by high correlations and interactions (see e.g., Boulesteix et al. [16], Ziegler and König [17]).”
Random forest versus logistic regression: a large-scale benchmark experiment (2018, background): “In practice, however, performance reaches a plateau with a few hundreds of trees for most datasets [18].”
Under review as a conference paper at ICLR 2020 many domain adaptation methods (2019, background): “…many recent works to improve the performance of random forests: Dheenadayalan et al. (2016) proposes pruning nodes for efficient learning, Ristin et al. (2015a) presents incremental modeling for large scale recognition, and Probst & Boulesteix (2017) investigates how to tune the number of trees.”
Forest aboveground biomass estimation using Landsat 8 and Sentinel-1A data with machine learning algorithms (2020, background): “In practice, it is controversial “whether the number of trees should simply to be set to the largest computationally manageable value or whether a smaller value may in some cases be better, in which case the number of trees should ideally be tuned carefully” 24 .”