Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking

By Michael Sejr Schlichtkrull, Nicola De Cao, Ivan Titov. International Conference on Learning Representations, 2020

298 citations recorded by the source corpus.

Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. However, there has been little work on interpreting them, and specifically on understanding which parts of the graphs (e.g. syntactic trees or co-reference structures) contribute to a prediction. In this work, we introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. Given a trained GNN model, we learn a simple classifier that, for every edge in every layer, predicts if that edge can be dropped. We demonstrate that such a classifier can be trained in a fully differentiable fashion, employing stochastic gates and encouraging sparsity through the expected $L_0$ norm. We use our technique as an attribution method to analyze GNN models for two tasks -- question answering and semantic role labeling -- providing insights into the information flow in these models. We show that we can drop a large proportion of edges without deteriorating the performance of the model, while we can analyse the remaining edges for interpreting model predictions.

View Paper: https://www.semanticscholar.org/paper/dcb5f0c180e138e3dae95d6e65237ef10063474c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

On Explainability of Graph Neural Networks via Subgraph Explorations (2021, methodology): “In addition, GraphMask (Schlichtkrull et al., 2020) follows a similar idea as PGExplainer that train a classifier to predict if an edge can be dropped without affecting model predictions.”
Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism (2022, methodology): “Besides the models (Ying et al., 2019; Luo et al., 2020; Schlichtkrull et al., 2021; Yu et al., 2021) that we have compared with in detail in Sec.”
Higher-Order Explanations of Graph Neural Networks via Relevant Walks (2020, methodology): “GraphMask [30] learns binary masks for edges in each layer to”
Explainability in Graph Neural Networks: A Taxonomic Survey (2020, methodology): “The other methods only need the forward computations of the trained GNNs. TABLE 1 A Comprehensive Analysis of Different Explanation Methods Method TYPE LEARNING TASK TARGET BLACK-BOX FLOW DESIGN SA [54], [55] Instance-level ✗ GC/NC N/E/NF ✗ Backward ✗ Guided BP [54] Instance-level ✗ GC/NC N/E/NF ✗ B”