Regularizing Black-box Models for Improved Interpretability

By Gregory Plumb, Maruan Al-Shedivat, Ángel Alexander Cabrera, Eric P. Xing, Ameet Talwalkar. 2019

85 citations recorded by the source corpus.

Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regularizes a model for explanation quality at training time. Importantly, these regularizers are differentiable, model agnostic, and require no domain knowledge to define. We demonstrate that post-hoc explanations for ExpO-regularized models have better explanation quality, as measured by the common fidelity and stability metrics. We verify that improving these metrics leads to significantly more useful explanations with a user study on a realistic task.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, background): “Another line of work has studied explanation regularization as an interpretable approach to impose prior knowledge into neural networks [38, 35, 36, 30, 7].”
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs (2020, unknown): “Approaches from the first category range from modifications of network architectures [67], over regularization of models encouraging interpretability [38,46], towards combinations of both [64].”
A Survey on Neural Network Interpretability (2020, methodology): “C. Active, Attribution as Explanation Similar to tree regularization which helps to achieve better global interpretability (decision trees), ExpO [88] added an interpretability regularizer in order to improve the quality of local attribution.”
RL-LIM: Reinforcement Learning-based Locally Interpretable Modeling (2019, methodology): “To be useful in practice, locally interpretable models need to maximize two objectives: (i) the overall prediction performance (how well it predicts compared to the ground truth labels) – for the model to be accurate, and (ii) fidelity (how well it approximates the ‘black-box’ model predictions) – t”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, unknown): “[129] proposed two metrics for measuring”
Human Evaluation of Models Built for Interpretability (2019, background): “…the works above that either do not empirically justify the interpretability of their approach through human-evaluation (e.g. Ustun and Rudin, 2016; Plumb et al., 2019) or that run user studies comparing their approach to base-lines, but do not attempt to yield generalizable insights (e.g.…”