Making Tree Ensembles Interpretable

By Satoshi Hara, Kohei Hayashi. 2016

73 citations recorded by the source corpus.

Tree ensembles, such as random forest and boosted trees, are renowned for their high prediction performance, whereas their interpretability is critically limited. In this paper, we propose a post processing method that improves the model interpretability of tree ensembles. After learning a complex tree ensembles in a standard way, we approximate it by a simpler model that is interpretable for human. To obtain the simpler model, we derive the EM algorithm minimizing the KL divergence from the complex ensemble. A synthetic experiment showed that a complicated tree ensemble was approximated reasonably as interpretable.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Local Rule-Based Explanations of Black Box Decision Systems (2018, methodology): “Decision trees have been adopted to globally explain neural networks [9, 22] and tree ensembles [18, 34].”
A Survey of Methods for Explaining Black Box Models (2018, background): “In Reference [38] the complexity is identified by the number of regions, i.”
Interpreting tree ensembles with inTrees (2014, background): “, reduced decision nodes referred to as node harvest [28], rules generated from simplified predictive regions [22,23], single decision trees [3,4,34, 36], and linear models [14].”
The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations (2019, methodology): “…used to approximate the decision function of the black-box and build understandable explanations, either using a surrogate model (e.g. a linear model [Ribeiro et al. , 2016] or a decision tree [Hara and Hayashi, 2016]), or by computing meaningful coefﬁcients [Turner, 2015; Lundberg and Lee, 2017]).”
A Survey on the Explainability of Supervised Machine Learning (2020, methodology): “Hara and Hayashi (2016) approximate a simple model from tree ensembles by deriving the expectation maximization algorithm minimizing the Kullback-Leibler divergence.”
Principles and Practice of Explainable Machine Learning (2020, background): “ion or feature relevance explanation categories. In the sequel, we will review some of the most popular approaches. 8.1.1 Simplifying and Extracting Rules ‹ An attempt to simplify RFs can be found in [33], where the authors propose a way to approximate them with a mixture of simpler models. The only”
Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence (2021, methodology): “Causal explanations can be obtained with methods such as causal modeling or approximate decision trees [37, 38, 39, 40].”
Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable (2016, background): “ domain experts despite their typically superior predictive capabilities [2]. Currently, attempts to interpret tree ensembles include seeking one tree that "best represents" a tree ensemble [7, 8, 9], model-agnostic (not exclusive to tree ensembles) explanations of predictions [10, 11] and”
Defining Locality for Surrogates in Post-hoc Interpretablity (2018, background): “Surrogate models (Craven & Shavlik, 1996; Hara & Hayashi, 2016), on the other hand, train an inter-pretable model (e.g. linear regression or decision tree with low complexity) to mimic the black-box decisions in order to extract explanations from it.”
Open the Black Box Data-Driven Explanation of Black Box Decision Systems (2018, methodology): “For example, decision trees have been adopted to explain neural networks [26] and tree ensembles [22, 42], while decision rules have been widely used to explain neural networks [1, 2] and support vector machines [16].”