Hierarchical Graph Representation Learning with Differentiable Pooling
By Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, Jure Leskovec. 2018
2,540 citations recorded by the source corpus.
Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction. However, current GNN methods are inherently flat and do not learn hierarchical representations of graphs---a limitation that is especially problematic for the task of graph classification, where the goal is to predict the label associated with an entire graph. Here we propose DiffPool, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion. DiffPool learns a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapping nodes to a set of clusters, which then form the coarsened input for the next GNN layer. Our experimental results show that combining existing GNN methods with DiffPool yields an average improvement of 5-10% accuracy on graph classification benchmarks, compared to all existing pooling approaches, achieving a new state-of-the-art on four out of five benchmark datasets.
Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Percy Liang
- Mohit Bansal
- Jure Leskovec
- Ruslan Salakhutdinov
- Sergey Levine
- Jianfeng Gao
- Quoc V. Le
In Information Systems
- Charu C. Aggarwal
- Philip S. Yu
- Jure Leskovec
- Marinka Zitnik
- Yoshua Bengio
- George Karypis
- Michael M. Bronstein
- Xiangnan He
In Statistical and Nonlinear Physics
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Philip S. Yu
- Marinka Zitnik
- Matthias Fey
- Rex Ying
- Michael M. Bronstein
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Stephan Günnemann
- Matthias Fey
- Michael M. Bronstein
- Rex Ying
- Marinka Zitnik
- Jian Tang
Citations in the corpus
- Object-Centric Learning with Slot Attention (2020, background): “[53, 58, 59]), with the exception of the Deep Set Prediction Network (DSPN) [31, 60], learn an ordered representation of the output set with learned per-element initialization, which prevents these approaches from generalizing to a different set cardinality at test time.”
- Strategies for Pre-training Graph Neural Networks (2019, methodology): “To obtain the entire graph’s representation h G , the READOUT function pools node features from the final iteration K , READOUT is a permutation-invariant function, such as averaging or a more sophisticated graph-level pooling function (Ying et al., 2018b; Zhang et al., 2018).”
- Fast Graph Representation Learning with PyTorch Geometric (2019, methodology): “We omit Jumping Knowledge when comparing global pooling operators, and hence report an additional Except for DiffPool (Ying et al., 2018), (global) pooling operators do not perform as benefically as expected to their respective (flat) counterparts, especially when baselines are enhanced by Jumping…”
- Position-aware Graph Neural Networks (2019, background): “…2017b) has led to advances on tasks such as node classification (Kipf & Welling, 2017), link prediction (Grover & Leskovec, 2016), graph classification (Ying et al., 2018b) and graph generation (You et al., 2018b), with successful applications across domains such as social and information networks…”
- Unsupervised Inductive Whole-Graph Embedding by Preserving Graph Proximity (2019, methodology): “…(Simonovsky & Komodakis, 2017), SET2SET (Gilmer et al., 2017), GRAPHSAGE, DGCNN/SORTPOOL (Zhang et al., 2018), SAN (Zhao et al., 2018), DIFFPOOL (Ying et al., 2018), CAPSGNN (Zhang & Chen, 2019), etc. Notice that most of these models adopt the neighbor aggregation based node embedding methods…”
- Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity (2019, methodology): “A large amount of graph neural network models resort to graph labels as a source of such information, making the models supervised aiming to improve graph classification accuracy specifically, such as DIFFPOOL [Ying et al., 2018], CAPSGNN [Zhang and Chen, 2019], etc.”
- How Powerful are Graph Neural Networks? (2018, methodology): “…the READOUT function aggregates node features from the final iteration to obtain the entire graph’s representation h G : (2.4) READOUT can be a simple permutation invariant function such as summation or a more sophisticated graph-level pooling function (Ying et al., 2018; Zhang et al., 2018).”
- Deep Graph Infomax (2018, methodology): “…readout to perform the best across all our experiments, we assume that its power will diminish with the increase in graph size, and in those cases, more sophisticated readout architectures such as set2vec (Vinyals et al., 2015) or DiffPool (Ying et al., 2018b) are likely to be more appropriate.”
- Provably Powerful Graph Networks (2019, neutral): “5 DiffPool (Ying et al., 2018) NA NA 78.”
- Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks (2018, methodology): “GNNs have achieved state-of-the-art performance on several graph classification benchmarks in recent years, see, e.g., (Ying et al. 2018b)—as well as applications such as protein-protein interaction prediction (Fout et al. 2017), recommender systems (Ying et al. 2018a), and the analysis of quantum…”
- Lifelong Graph Learning (2020, neutral): “edding by sampling and aggregating neighborhood features. JK-Net [38] followed the same sampling strategy and demonstrated a significant accuracy improvement on GCN with more than two layers. DiffPool [39] learned a differentiable soft cluster assignment to map nodes to a set of clusters, which then ”
- A simple yet effective baseline for non-attribute graph classification (2018, background): “Many GNN variants with different neighborhood aggregation and graph-level pooling schemes have been proposed [DBV16, DMI + 15, HYL17, LTBZ15, KMB + 16, KW16, VCC + 17, YYM + 18].”
- Towards Sparse Hierarchical Graph Classifiers (2018, methodology): “The main insight by recent works [28, 1] is that intermediate node representations (e.”
- A Comprehensive Survey on Graph Neural Networks (2019, background): “Recently, a differentiable pooling (DiffPool) [54] is proposed, which can generate hierarchical representations of graphs.”
- Permutation Invariant Graph Generation via Score-Based Generative Modeling (2020, methodology): “In particular, we implicitly define a per-Permutation As one future direction, we hope to improve the scalability of our model by reducing the computational complexity, using techniques such as graph pooling (Ying et al., 2018).”
- Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks (2019, background): “This kind of approach hinders effective classification over entire graphs [23, 24].”
- Language is All a Graph Needs (2023, background): “Both SuperGAT (Kim and Oh, 2022) and DiffPool (Ying et al., 2018) introduce auxiliary link prediction task, thus successfully obtain better node representations and performance for node or graph classification, demonstrating that model’s comprehension of graph structure can be significantly enhanced”
- MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing (2019, background): “Other recent work like (Ying et al., 2018), which focuses on hierarchical pooling for community-aware graph representation might also be extended to use general neighborhood mixing layers.”