Geom-GCN: Geometric Graph Convolutional Networks

By Hongbin Pei, Bingzhen Wei, Kevin Chen-Chuan Chang, Yu Lei, Bo Yang. International Conference on Learning Representations, 2020

1,493 citations recorded by the source corpus.

Message-passing neural networks (MPNNs) have been successfully applied in a wide variety of applications in the real world. However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs.

View Paper: https://www.semanticscholar.org/paper/04f3203f1214063436d81ce0c2ad7623204da488

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Xiao Wang
2. Yoshua Bengio
3. Yao Ma
4. Sergey Levine
5. Oleg Platonov
6. Dacheng Tao
7. Yi Yang
8. Jiliang Tang

Citations in the corpus

• How Powerful are Spectral Graph Neural Networks (2022, methodology): “We also use heterogeneous graphs, including Wikipedia graphs Chameleon and Squirrel (Rozemberczki et al., 2021), the Actor co-occurrence graph, and the webpage graph Texas and Cornell from WebKB3 (Pei et al., 2020).”
• Parameter Prediction for Unseen Deep Architectures (2021, background): “[63] Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang.”
• Is Homophily a Necessity for Graph Neural Networks? (2021, methodology): “For all datasets, we follow the experimental setting provided in [26], which consists of 10 random splits with proportions 48/32/20% corresponding to training/validation/test for each graph.”
• BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation (2021, methodology): “We also include the Wikipedia graph Chameleon and Squirrel [26], the Actor co-occurrence graph, and webpage graphs Texas and Cornell from WebKB‡ [22].”
• Long Range Graph Benchmark (2022, unknown): “Similarly, some recent models have been designed to perform non-local feature integration in particular aspect of non-homophilic graphs [49, 40].”
• Simple and Deep Graph Convolutional Networks (2020, methodology): “Following the setting in (Pei et al., 2020), we use 7 datasets: Cora, Citeseer, Pubmed, Chameleon, Method PPI GraphSAGE (Hamilton et al., 2017) 61.2 VR-GCN (Chen et al., 2018b) 97.8 GaAN (Zhang et al., 2018) 98.71 GAT (Veliˇckovi´c et al., 2018) 97.3 JKNet (Xu et al., 2018) 97.6 GeniePath (Liu et…”
• Towards Deeper Graph Neural Networks (2020, background): “Recently, GeomGCN [25] and non-local GNNs [16] are proposed to capture longrange dependencies for disassortative graph by designing non-local aggregators.”
• Language is All a Graph Needs (2023, methodology): “…(He et al., 2023), we select those top-ranked GNNs from the two corresponding benchmarks with 60%/20%/20% train/val/test splits, including Snowball, RevGAT, etc. (Abu-El-Haija et al., 2019; Pei et al., 2020; Wu et al., 2019; He et al., 2021; Bo et al., 2021; Chen et al., 2020b; Luan et al., 2022).”
• A Survey on Oversmoothing in Graph Neural Networks (2023, unknown): “Additionally, we test the qualitative behavior of those measures on three different graph datasets, i.e., small-scale Texas graph, medium-scale Cora graph, and large-scale Cornell5 graph, and observe an exponential convergence to zero of all measures for standard GNN models (i.e., GCN, GAT, and Grap”
• Finding Global Homophily in Graph Neural Networks When Meeting Heterophily (2022, methodology): “by (Pei et al., 2020) and 6 large-scale datasets from (Lim et al.”
• Beyond Low-frequency Information in Graph Convolutional Networks (2021, methodology): “GraphSAGE (Hamilton, Ying, and Leskovec 2017) designs a permutation-invariant aggregator for message passing; GAT (Velickovic et al. 2018) employs self-attention to calculate the coefficients of neighbors in aggregation; MoNet (Monti et al. 2017) provides a unified generalization of graph convolutiona”
• Implicit Graph Neural Networks (2020, methodology): “Recently, global method Geom-GCN (Pei et al., 2020) employs additional embedding approaches to capture global information.”
• Understanding over-squashing and bottlenecks on graphs via curvature (2021, background): “As well as reporting results we give a measure of homophilyH(G) proposed by Pei et al. (2019) (restated in Appendix F, equation 21), by which we can see our experiment set is diverse with respect to homophily.”
• A critical look at the evaluation of GNNs under heterophily: are we really making progress? (2023, methodology): “Pei et al. (2020) were the first to attract attention to learning under heterophily.”
• Heterogeneous Graph Structure Learning for Graph Neural Networks (2021, methodology): “…graph embedding methods, i.e., MP2Vec (Dong, Chawla, and Swami 2017), HAN (Wang et al. 2019b), HeGAN (Hu, Fang, and Shi 2019), and GTN (Yun et al. 2019), and three graph structure learning related methods, i.e. LDS (Franceschi et al. 2019), Pro-GNN (Jin et al. 2020), and Geom-GCN (Pei et al. 2020).”
• Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs (2020, methodology): “For all benchmarks (except Cora-Full), we use the feature vectors, class labels, and 10 random splits (48%/32%/20% of nodes per class for train/validation/test2) provided by [26].”
• Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, methodology): “[58] (and collected by [61, 66, 48]); see our Table 1 for statistics.”