Analyzing the Interpretability Robustness of Self-Explaining Models

By Haizhong Zheng, Earlence Fernandes, Atul Prakash. 2019

7 citations recorded by the source corpus.

Recently, interpretable models called self-explaining models (SEMs) have been proposed with the goal of providing interpretability robustness. We evaluate the interpretability robustness of SEMs and show that explanations provided by SEMs as currently proposed are not robust to adversarial inputs. Specifically, we successfully created adversarial inputs that do not change the model outputs but cause significant changes in the explanations. We find that even though current SEMs use stable co-efficients for mapping explanations to output labels, they do not consider the robustness of the first stage of the model that creates interpretable basis concepts from the input, leading to non-robust explanations. Our work makes a case for future work to start examining how to generate interpretable basis concepts in a robust way.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “Robustness Analysis Robustness of post-hoc ex-tractive interpretability methods has been studied (Kindermans et al., 2019; Ghorbani et al., 2019; Heo et al., 2019; Zheng et al., 2019; Slack et al., 2020).”