TIP: Typifying the Interpretability of Procedures

By Amit Dhurandhar, Vijay Iyengar, Ronny Luss, Karthikeyan Shanmugam. 2017

35 citations recorded by the source corpus.

We provide a novel notion of what it means to be interpretable, looking past the usual association with human understanding. Our key insight is that interpretability is not an absolute concept and so we define it relative to a target model, which may or may not be a human. We define a framework that allows for comparing interpretable procedures by linking it to important practical aspects such as accuracy and robustness. We characterize many of the current state-of-the-art interpretable methods in our framework portraying its general applicability. Finally, principled interpretable strategies are proposed and empirically evaluated on synthetic data, as well as on the largest public olfaction dataset that was made recently available \cite{olfs}. We also experiment on MNIST with a simple target model and different oracle models of varying complexity. This leads to the insight that the improvement in the target model is not only a function of the oracle models performance, but also its relative complexity with respect to the target model.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, neutral): “[167], Puri et al.”
Asking 'Why' in AI: Explainability of intelligent systems - perspectives and challenges (2018, background): “The difference between explanations generated for human consumption and those generated for machine consumption is an area that researchers are beginning to consider (Dhurandhar et al., 2017); ultimately, however, it is likely that future AI systemswill need to provide both kinds of explanation.”
Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives (2018, background): “Moreover, there is also some interesting work which tries to formalize and quantify interpretability [10].”
Interpretable Counterfactual Explanations Guided by Prototypes (2019, background): “This is different from (Dhurandhar et al. 2017) who define an interpretability metric relative to a target model.”
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems (2018, background): “Dhurandhar et al. developed, to our knowledge, the ﬁrst formal, quantiﬁable deﬁnition of interpretability (Dhurandhar et al., 2017b;a), which proposes measuring the interpretability of a procedure in relation to the performance of a target model (which could be a human or non-human agent) on a…”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, background): “Instead of having one groundtruth explanation, various quantifiable aspects of interpretability are emerging [87,86,77,46,131,3,102,87,21,31].”