Explaining Deep Learning Models using Causal Inference

By Tanmayee Narendra, Anush Sankaran, Deepak Vijaykeerthy, Senthil Mani. 2018

59 citations recorded by the source corpus.

Although deep learning models have been successfully applied to a variety of tasks, due to the millions of parameters, they are becoming increasingly opaque and complex. In order to establish trust for their widespread commercial use, it is important to formalize a principled framework to reason over these models. In this work, we use ideas from causal inference to describe a general framework to reason over CNN models. Specifically, we build a Structural Causal Model (SCM) as an abstraction over a specific aspect of the CNN. We also formulate a method to quantitatively rank the filters of a convolution layer according to their counterfactual importance. We illustrate our approach with popular CNN architectures such as LeNet5, VGG19, and ResNet32.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Causal Interpretability for Machine Learning - Problems, Methods and Evaluation (2020, background): “is type of models is still an open problem. One common way of evaluating such models is to report the most important components of a model by measuring their causal eﬀects on the outcome of the model [38; 77]. Chattopadhyay et al. also used the causal attribution of each neuron on the output to visu”
HARK Side of Deep Learning - From Grad Student Descent to Automated Machine Learning (2019, background): “Similarly, use of causal inference has recently been shown to be promising in understanding underlying mechanisms of deep learning systems [76] and if descriptive, causal modals can answer prediction, intervention and counterfactual questions [77].”
Evaluating Explanation Without Ground Truth in Interpretable Machine Learning (2019, background): “Furthermore, as for the posthoclocal explanations in form of training samples [18] and model components [26], ablation and perturbation operations are properly applied as well in evaluating the explanation fidelity.”
Generative causal explanations of black-box classifiers (2020, background): “Other work generates causal explanations by performing interventions in different network layers [49], using latent factors built into a modified network architecture [38], or using labeled examples of human-interpretable latent factors [50].”