Interpretable & Explorable Approximations of Black Box Models

By Himabindu Lakkaraju, Ece Kamar, R. Caruana, Jure Leskovec. 2017

259 citations recorded by the source corpus.

We propose Black Box Explanations through Transparent Approximations (BETA), a novel model agnostic framework for explaining the behavior of any black-box classifier by simultaneously optimizing for fidelity to the original model and interpretability of the explanation. To this end, we develop a novel objective function which allows us to learn (with optimality guarantees), a small number of compact decision sets each of which explains the behavior of the black box model in unambiguous, well-defined regions of feature space. Furthermore, our framework also is capable of accepting user input when generating these approximations, thus allowing users to interactively explore how the black-box model behaves in different subspaces that are of interest to the user. To the best of our knowledge, this is the first approach which can produce global explanations of the behavior of any given black box model through joint optimization of unambiguity, fidelity, and interpretability, while also allowing users to explore model behavior based on their preferences. Experimental evaluation with real-world datasets and user studies demonstrates that our approach can generate highly compact, easy-to-understand, yet accurate approximations of various kinds of predictive models compared to state-of-the-art baselines.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Sanity Checks for Saliency Maps (2018, background): “Explanations could potentially help satisfy regulatory requirements [3], help practitioners debug their model [4, 5], and perhaps, reveal bias or other unintended effects learned by a model [6, 7].”
Learning Optimal Decision Trees with SAT (2018, background): “, 2017], decision sets [Lakkaraju et al., 2017], and sums of terms (i.”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, methodology): “[29] introduces a quantifiable measure of clarity (unambiguity) for their method.”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “In [54] the authors propose a method f exploiting a two-level boolean rule predictor to”
An Interpretable Model with Globally Consistent Explanations for Credit Risk (2018, background): “they provide useful patterns, which has been a popular form of explanation in the state-of-the-art literature on model explanations [7, 4, 13].”
Manipulating and Measuring Model Interpretability (2018, background): “[63] Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec.”
Interpreting Blackbox Models via Model Extraction (2017, methodology): “Next, we compare the fidelity of our decision trees to that of rule lists [33] and decision sets [19, 20] (based on the largest size decision trees for each dataset).”
The challenge of crafting intelligible intelligence (2018, background): “Lakkaraju et al. [22] suggest formulating an explicit optimization of this form and propose an approximation algorithm for generating global explanations in the form of compact sets of if-then rules.”
Explaining Explanations in AI (2018, background): “A trade-off inherently occurs between the insightfulness of the approximated model, the simplicity of the presented function, and the size of the domain to which is applies and remains valid (Bastani et al., 2017; Lakkaraju et al., 2017).”
Improving the Interpretability of Deep Neural Networks with Knowledge Distillation (2018, background): “Past research focus on either global interpretations [22] [23] or local explanations [24] [25] [26].”
iForest: Interpreting Random Forests via Visual Analytics (2019, background): “Surrogate models are often simple and interpretable, such as decision trees [48, 60], decision rules [16, 23], and decision sets [32].”
On Explaining Decision Trees (2020, unknown): “[2,68,25,32,1,7,57,24,26,70,67,58,44,30,17,5,73,35,4,78,69,10,34,72,71,36,46,11,45].”
A Survey on the Explainability of Supervised Machine Learning (2020, methodology): “The first level specifies the neighborhood of a rule (subspace descriptor) and the second level introduces decision rules that are specific for each region (Lakkaraju et al., 2017).”