Explainability Methods for Graph Convolutional Neural Networks

By Phillip E. Pope, Soheil Kolouri, Mohammad Rostami, Charles E. Martin, Heiko Hoffmann. 2019

586 citations recorded by the source corpus.

With the growing use of graph convolutional neural networks (GCNNs) comes the need for explainability. In this paper, we introduce explainability methods for GCNNs. We develop the graph analogues of three prominent explainability methods for convolutional neural networks: contrastive gradient-based (CG) saliency maps, Class Activation Mapping (CAM), and Excitation Back-Propagation (EB) and their variants, gradient-weighted CAM (Grad-CAM) and contrastive EB (c-EB). We show a proof-of-concept of these methods on classification problems in two application domains: visual scene graphs and molecular graphs. To compare the methods, we identify three desirable properties of explanations: (1) their importance to classification, as measured by the impact of occlusions, (2) their contrastivity with respect to different classes, and (3) their sparseness on a graph. We call the corresponding quantitative metrics fidelity, contrastivity, and sparsity and evaluate them for each method. Lastly, we analyze the salient subgraphs obtained from explanations and report frequently occurring patterns.

Subfields: Artificial Intelligence, Materials Chemistry, Graph Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Graph Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Informativeness [5, 44, 21, 25, 26, 45, 30, 32, 34, 35, 37, 38, 41, 46, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 86, 87, 88, 89, 59, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 1”
On Explainability of Graph Neural Networks via Subgraph Explorations (2021, methodology): “Specifically, we employ the metrics Fidelity and Sparsity to evaluate explanation results (Pope et al., 2019; Yuan et al., 2020c).”
BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis (2020, methodology): “works have been used for explaining GNNs using post-hoc interpretation meth526 ods [35,3,53].”
Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism (2022, background): “the input features to find important features (Pope et al., 2019; Baldassarre & Azizpour, 2019).”
Machine Learning for High-entropy Alloys: Progress, Challenges and Opportunities (2022, methodology): “The most popular CNN model for atomic interactions is GCNN, which demonstrates excellent interpretability [235, 239] by maintaining the concepts of atoms and chemical bonds.”
Machine Learning Interpretability: A Survey on Methods and Metrics (2019, methodology): “There are CVPR 2019 papers with a great focus on interpretability and explainability for computer vision, such as explainability methods for graph CNNs [138], interpretable and fine-grained visual explanations for CNNs [139], interpreting CNNs via Decision Trees [140], and learning to explain with c”
Graph Neural Network for Traffic Forecasting: A Survey (2021, methodology): “The explainability study for graph neural networks in other studies is still in the early stage [248], [249], [250] and it remains a challenge how these explainability techniques can be used in the traffic forecasting domain.”
Drug discovery with explainable artificial intelligence (2020, background): “59 Pope et al. 62 adapted gradient-based feature at-tribution 91,92 for the identiﬁcation of relevant functional groups in adverse eﬀect prediction.”
PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks (2020, background): “Selecting E in explaining GNN’s prediction can vary from a subset of edges of G [24] to a subset of entries in X [27].”
Explainability in Graph Neural Networks: A Taxonomic Survey (2020, methodology): “Grad-CAM [55] extends the CAM to general graph classification models by removing the constraint of the GAP layer.”
BERT Learns (and Teaches) Chemistry (2020, methodology): “This work was an extension of their 2019 paper on explainability methods for GCNs [22] and proposed the application of contrastive gradient-based saliency maps, class activation mapping (CAM), and excitation backpropagation, previously proposed for convolutional neural networks, to the graph…”
Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking (2020, result): “Furthermore, in contrast to techniques based on artificial gradients (Pope et al., 2019; Xie & Lu, 2019; Schwarzenberg et al., 2019), erasure search would provide implementation invariance (Sundararajan et al.”
Parameterized Explainer for Graph Neural Network (2020, methodology): “(4) We ﬁrst adopt Gradient in [38] to calculate the importance of each node, then calculate the importance of an edge by average the connected nodes’ importance scores.”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “Recently, more researches focus on the interpretations of GNN models, such as GraphLIME [76], CoGE [51], Counterfactual explanations on GNNs [18] and others [20, 111, 132].”
Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI (2021, background): “With the increased adoption of graph convolutional neural networks (GCNNs) [33], explainability methods for GCNNs have been also introduced recently [70].”