Global Model Interpretation Via Recursive Partitioning

By Chengliang Yang, Anand Rangarajan, Sanjay Ranka. 2018

89 citations recorded by the source corpus.

In this work, we propose a simple but effective method to interpret black-box machine learning models globally. That is, we use a compact binary tree, the interpretation tree, to explicitly represent the most important decision rules that are implicitly contained in the black-box machine learning models. This tree is learned from the contribution matrix which consists of the contributions of input variables to predicted scores for each single prediction. To generate the interpretation tree, a unified process recursively partitions the input variable space by maximizing the difference in the average contribution of the split variable between the divided spaces. We demonstrate the effectiveness of our method in diagnosing machine learning models on multiple tasks. Also, it is useful for new knowledge discovery as such insights are not easily identifiable when only looking at single predictions. In general, our work makes it easier and more efficient for human beings to understand machine learning models.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mengzhou Li
2. Jinjun Xiong
3. Wei Cheng
4. Nadia Burkart
5. Dongsheng Luo
6. Arjan Durresi
7. Marco F. Huber
8. Ge Wang

Citations in the corpus

• Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “[80] proposed a global model interpretation via recursive partitioning called (GIRP) to build a global interpretation tree for a wide range of ML models based on their local explanations.”
• A Survey on the Explainability of Supervised Machine Learning (2020, methodology): “Yang et al. (2018a) propose a binary decision tree that represents the most important decision rules.”
• Parameterized Explainer for Graph Neural Network (2020, methodology): “Although explanations provided by the leading method GNNExplainer [53] preserve the local fidelity, they do not help to understand the general picture of the model across a population [50].”
• Trustworthy Artificial Intelligence: A Review (2022, background): “It is mainly used for applications that predict global population trends like climate change [219].”
• On Interpretability of Artificial Neural Networks: A Survey (2020, methodology): “The gist of direct extraction is to construct a new interpretable model such as a decision tree [92], [192] or a rule-based system directly from the trained model.”