Towards Explanation of DNN-based Prediction with Guided Feature Inversion

By Mengnan Du, Ninghao Liu, Qingquan Song, Xia Hu. 2018

134 citations recorded by the source corpus.

While deep neural networks (DNN) have become an effective computational tool, the prediction results are often criticized by the lack of interpretability, which is essential in many real-world applications such as health informatics. Existing attempts based on local interpretations aim to identify relevant features contributing the most to the prediction of DNN by monitoring the neighborhood of a given input. They usually simply ignore the intermediate layers of the DNN that might contain rich information for interpretation. To bridge the gap, in this paper, we propose to investigate a guided feature inversion framework for taking advantage of the deep architectures towards effective interpretation. The proposed framework not only determines the contribution of each feature in the input but also provides insights into the decision-making process of DNN models. By further interacting with the neuron of the target category at the output layer of the DNN, we enforce the interpretation result to be class-discriminative. We apply the proposed interpretation model to different CNN architectures to provide explanations for image data and conduct extensive experiments on ImageNet and PASCAL VOC07 datasets. The interpretation results demonstrate the effectiveness of our proposed framework in providing class-discriminative interpretation for DNN-based prediction.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks (2019, methodology): “The resulting explanations (Fig 1 b) are valid model inputs and their faithfulness can, thus, be directly verified (as in methods from [17, 14, 6, 11]).”
DARPA's Explainable Artificial Intelligence (XAI) Program (2019, methodology): “The TAMU system handles image (Du et al. 2018) and text (Gao et al. 2017) data and is being applied to the XAI analytics challenge problem area.”
Learning Credible Deep Neural Networks with Rationale Regularization (2019, methodology): “, x ) n } using three sizes of kernel: [2, 3, 4].”
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks (2019, methodology): “s its application. Grad-CAM [30] and Grad-CAM++ [6] generalize CAM [41] and mainly differ in how to calculate the weights of activation maps. Extension of such approaches have been proposed, [30] and [9] combine Class Activation Mapping with Backpropagation and Perturbation respectively. Recently, S”
Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment (2019, methodology): “Inverting a neural network is one important method to understand the representations learned by neural networks [19, 23, 29, 54, 66, 77].”
Evaluating Explanation Without Ground Truth in Interpretable Machine Learning (2019, background): “rial perturbations, especially for those posthoc ones from neural architectures (62). Explanation robustness is designed to measure how similar the explanations are for similar instances. Recent work (25,63,64) all conduct robustness evaluation with metrics on sensitivity, beyond the evaluation on p”
On Explainability of Graph Neural Networks via Subgraph Explorations (2021, background): “Recently, extensive efforts have been made to study explanation techniques for deep models on images and text (Simonyan et al., 2013; Yuan et al., 2019; Smilkov et al., 2017; Yuan et al., 2020a; Yang et al., 2019; Du et al., 2018).”
Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment (2019, methodology): “[70] explain a DNN-based prediction by a guided feature inversion framework to identify the contribution of each feature in the input as well as to inspect the information employed by the DNN for the prediction task.”
Techniques for interpretable machine learning (2018, background): “capture the high-level content of input images as well as their spatial arrangement, a guided feature inversion framework is proposed to provide local explanations [11].”
Explainability in Graph Neural Networks: A Taxonomic Survey (2020, methodology): “Several techniques are proposed to explain the deep models for image and text data [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21].”
dEFEND: Explainable Fake News Detection (2019, background): “Therefore, the explanation for deep neural networks (DNNs) mainly focuses on understanding the representations captured by neurons at intermediate layers of DNNs [9, 26, 28].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “[52] evaluated saliency maps generated from a CNN model by calculating pixel-wise IOU (intersection over union) of model-explanation bounding boxes and ground truth bounding boxes.”
A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning (2018, methodology): “In another work, Du et al. (2018) calculate the mAP between the bounding boxes of an objects’ saliency mask and the ground truth bounding boxes to evaluate their interpretability technique as an”
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models (2020, background): “Additionally, it is not guaranteed that the resulting explanations are faithful and reﬂect the decision making process of the model [14, 49, 58].”
Interpretable Deep Learning under Fire (2018, background): “Specifically, we compare the interpretations of benign and adversarial inputs, which is crucial for understanding the security implications of using interpretability as a means of defenses [13, 57].”
Explainable Artificial Intelligence: a Systematic Review (2020, background): “Similarly, Guided Feature Inversion [202] generates an inversion image representation consisting of the weighted sum between the original image and another noisy background image, such as a grey-scale image with each pixel set to an average colour, a Gaussian white noise or a blurred image.”