Interpreting Models via Single Tree Approximation

By Yichen Zhou, Giles Hooker. 2016

56 citations recorded by the source corpus.

We propose a procedure to build a decision tree which approximates the performance of complex machine learning models. This single approximation tree can be used to interpret and simplify the predicting pattern of random forests (RFs) and other models. The use of a tree structure is particularly relevant in medical questionnaires where it enables an adaptive shortening of the questionnaire, reducing response burden. We study the asymptotic behavior of splits and introduce an improved splitting method designed to stabilize tree structure. Empirical studies on both simulation and real data sets illustrate that our method can simultaneously achieve high approximation power and stability.

Subfields: Artificial Intelligence, Statistics and Probability, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Safety, Robustness & Interpretability

Citations in the corpus

Interpreting tree ensembles with inTrees (2014, background): “, reduced decision nodes referred to as node harvest [28], rules generated from simplified predictive regions [22,23], single decision trees [3,4,34, 36], and linear models [14].”
Benchmarking and survey of explanation methods for black box models (2021, methodology): “STA, Single Tree Approximation [132], is a specific global post-hoc explanation method tailored for random forests, in which the decision tree, used as an explanation, is constructed by exploiting test hypothesis to find the best splits.”
Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models (2020, methodology): “In this vein, approximations on the entire input space are provided in (Craven & Shavlik, 1996) and (Zhou & Hooker, 2016) among others, while LIME (Ribeiro et al., 2016) and its extension using decision rules (Ribeiro, Singh, & Guestrin, 2018) rely on this technique for providing local…”
iForest: Interpreting Random Forests via Visual Analytics (2019, background): “Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
Trustworthy AI: From Principles to Practices (2021, methodology): “been proposed to approximate ML models, such as random forests [317, 392] and neural networks [28, 86, 393].”
Principles and Practice of Explainable Machine Learning (2020, methodology): “On the other hand, the approach in [90] was based on studying the asymptotic behaviour of the Gini index, in order to train a stable and accurate decision tree.”
Born-Again Tree Ensembles (2020, background): “Recently, some works focused on directly extracting a single tree from a tree ensemble based on stabilized but yet heuristic splitting criteria (Zhou & Hooker, 2016), genetic algorithms (Van-dewiele et al., 2017), or by actively sampling training points (Bastani et al., 2017a;b).”