Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance

By Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin. 2016

74 citations recorded by the source corpus.

At the core of interpretable machine learning is the question of whether humans are able to make accurate predictions about a model's behavior. Assumed in this question are three properties of the interpretable output: coverage, precision, and effort. Coverage refers to how often humans think they can predict the model's behavior, precision to how accurate humans are in those predictions, and effort is either the up-front effort required in interpreting the model, or the effort required to make predictions about a model's behavior. In this work, we propose anchor-LIME (aLIME), a model-agnostic technique that produces high-precision rule-based explanations for which the coverage boundaries are very clear. We compare aLIME to linear LIME with simulated experiments, and demonstrate the flexibility of aLIME with qualitative examples from a variety of domains and tasks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Explanations of model predictions with live and breakDown packages (2018, methodology): “An interesting example of a situation in which trust issues are fully justiﬁed is presented in Tulio Ribeiro et al. (2016b).”
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “Recent examples are found in (Ribeiro et al., 2016a; Lakkaraju et al., 2017).”
A Survey of Methods for Explaining Black Box Models (2018, background): “The main intuition of LIME is that the explanation may be derived locally from the records generated randomly in the neighborhood of the record to be explained, and weighted according to their proximity to it.”
Did the Model Understand the Question? (2018, methodology): “One could also use model agnostic techniques like Ribeiro et al. (2016b).”
A Symbolic Approach to Explaining Bayesian Network Classifiers (2018, methodology): “More related to our work is the notion of an “anchor” introduced in [Ribeiro et al. , 2016a; Ribeiro et al. , 2018].”
Learning Interpretable Spatial Operations in a Rich 3D Blocks World (2017, background): “Unfortunately, it is unclear what models are learning and how much they are understanding versus memorizing bias in the training data (Ribeiro, Singh, and Guestrin 2016).”