Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach

By Satoshi Hara, Kohei Hayashi. 2016

100 citations recorded by the source corpus.

Tree ensembles, such as random forests and boosted trees, are renowned for their high prediction performance. However, their interpretability is critically limited due to the enormous complexity. In this study, we present a method to make a complex tree ensemble interpretable by simplifying the model. Specifically, we formalize the simplification of tree ensembles as a model selection problem. Given a complex tree ensemble, we aim at obtaining the simplest representation that is essentially equivalent to the original one. To this end, we derive a Bayesian model selection algorithm that optimizes the simplified model while maintaining the prediction performance. Our numerical experiments on several datasets showed that complicated tree ensembles were reasonably approximated as interpretable.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Interpreting tree ensembles with inTrees (2014, background): “, reduced decision nodes referred to as node harvest [28], rules generated from simplified predictive regions [22,23], single decision trees [3,4,34, 36], and linear models [14].”
Born-Again Tree Ensembles (2020, background): “Similarly, Sirikulviriya & Sinthupinyo (2011) focused on deducing rules from a random forest, while Hara & Hayashi (2016) focused on rule extraction from tree ensembles via bayesian model selection, and Mol-las et al. (2019) used a local-based, path-oriented similarity metric to select rules from a…”
Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable (2016, background): “Current attempts to interpret tree ensembles include seeking one tree that best represents the ensemble [18, 54], modelagnostic explanations not exclusive to tree ensembles [43], feature importance [20], partial dependence plots [11], etc.”
Human-in-the-loop machine learning: a state of the art (2022, background): “For tree ensembles, the additional layers of explanation found in the literature are explanation by simplification and feature relevance techniques (Hara and Hayashi 2018).”
Explainable artificial intelligence: a comprehensive review (2021, methodology): “For example, Hara and Hayashi (2018) solved the simplification of the tree ensemble algorithms by making a simplified model that is similar to the original model while preserving the original model?s performance.”
Explainable Artificial Intelligence: a Systematic Review (2020, methodology): “A similar approach is at the basis of the Factorized Asymptotic Bayesian (FAB) inference method [295] which consists of a bayesian model selection algorithm that simplified and optimized a tree ensemble.”
From hype to reality: data science enabling personalized medicine (2018, background): “The same general concern applies to approaches, in which a complex machine learning model is approximated by a simpler one to enhance interpretability, for example, using a decision tree [41, 42].”