Towards Robust Interpretability with Self-Explaining Neural Networks

By David Alvarez-Melis, Tommi S. Jaakkola. 2018

1,140 citations recorded by the source corpus.

Most recent work on interpretability of complex machine learning models has focused on estimating a posteriori explanations for previously trained models around specific predictions. Self-explaining models where interpretability plays a key role already during learning have received much less attention. We propose three desiderata for explanations in general – explicitness, faithfulness, and stability – and show that existing methods do not satisfy them. In response, we design self-explaining models in stages, progressively generalizing linear classifiers to complex yet architecturally explicit models. Faithfulness and stability are enforced via regularization specifically tailored to such models. Experimental results across various benchmark datasets show that our framework offers a promising direction for reconciling model complexity and interpretability.

Subfields: Artificial Intelligence, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, methodology): “MIPS can often be efficiently scaled using approximate algorithms (Shrivastava and Li, 2014) specific losses have been shown to improve inherently interpretable models (Melis and Jaakkola, 2018) for local interpretability.”
Measuring Association Between Labels and Free-Text Rationales (2020, background): “Self-Rationalizing Model (I→OR) A joint, self-rationalizing model (Melis and Jaakkola, 2018), illustrated in Figure 3, predicts both a label and rationale during inference.”
EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction (2019, background): “Closer to our work, [2] learn a self-explaining classiﬁer that takes as input a set of concepts extracted from the original input and a relevance score for each concept.”
Concept Bottleneck Models (2020, methodology): “As with standard models, SENN does not use any pre-specified concepts; it learns an input representation encouraged to be interpretable through diversity and smoothness constraints.”
Interpretability Beyond Classification Output: Semantic Bottleneck Networks (2019, neutral): “Similar to our proposed approach, [1, 20] and [16] embed an interpretable layer into the network, yet are different in the way that they treat the concept representations as free parameters and train them jointly with the network parameters.”
The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations (2019, background): “The stability of post-hoc explainer systems has been criticized as well, showing that some of these approaches are locally not stable enough [Alvarez Melis and Jaakkola, 2018] or on the contrary too stable and thus not locally accurate enough [ Laugel et al. , 2018b].”
Is Attention Interpretable? (2019, background): “(2017) and Melis and Jaakkola (2018), but merely as an alternative ordering method.”
On the Robustness of Interpretability Methods (2018, background): “A slight generalization of criterion (1) can be used to train interpretable neural networks with robust explanations (Alvarez-Melis & Jaakkola, 2018).”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “Accountability Auditability of algorithms/data/design Factsheets (lineage) Minimize and report negative impacts Fairness checklists Acknowledge and evaluate trade-offs Reporting the robustness-accuracy [1] trade-off or simplicity-equity trade-off [37] Ensure redress Fairness checklists -”
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects (2020, background): “stability of both gradient-based [8, 54, 66, 122] and counterfactualbased [104, 106] explanation systems has been called into question.”
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics (2021, background): “Example-based explanations provide a summarised overview of a model using representative examples or high-level concepts [80,90,91].”
Black Box Explanation by Learning Image Exemplars in the Latent Feature Space (2019, methodology): “18 As in [21], in our experiments, we use =0.”
Regularizing Black-box Models for Improved Interpretability (2019, methodology): “First, self-explaining neural networks (SENN) (Alvarez-Melis and Jaakkola, 2018a) (a variation of contextual explanation networks (CEN) (Al-Shedivat et al., 2017)) is an interpretable by-design approach that additionally (indirectly) optimizes their models to produce stable explanations.”
Trustworthy AI: From Principles to Practices (2021, methodology): “For example, previous research has embedded linear models and prototype selection to the DNN [16, 20, 73].”