Benchmarking Graph Neural Networks

By Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, Xavier Bresson. 2023

1,223 citations recorded by the source corpus.

Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. As the field grows, it becomes critical to identify key architectures and validate new ideas that generalize to larger, more complex datasets. Unfortunately, it has been increasingly difficult to gauge the effectiveness of new models in the absence of a standardized benchmark with consistent experimental settings. In this paper, we introduce a reproducible GNN benchmarking framework, with the facility for researchers to add new models conveniently for arbitrary datasets. We demonstrate the usefulness of our framework by presenting a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs) for a variety of graph tasks, i.e. graph regression/classification and node/link prediction, with medium-scale datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, methodology): “For the ZINC-10K dataset [74], we use the processed data in Pytorch Geometric format that is made available online 4 by the authors.”
Long Range Graph Benchmark (2022, neutral): “However, if the graph size is small, such as ogbg-mol* [27] or ZINC [14] datasets with average graph size in the range of 23-26 nodes, then r would effectively be small as well.”
Recipe for a General, Powerful, Scalable Graph Transformer (2022, methodology): “We compare GPS against a set of popular message-passing neural networks (GCN, GIN, GatedGCN, PNA, etc.), graph transformers (SAN, Graphormer, etc.), and other recent graph neural networks with SOTA results (CIN, CRaWL, GIN-AK+, ExpC).”
Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, background): “The small size of these datasets may tend to create models that are more prone to overfitting [21], which prevents the scaling up of GNNs designed for non-homophilous settings.”
Graph Neural Networks with Learnable Structural and Positional Representations (2021, methodology): “For evaluation and reporting of results, we follow the respective protocols as specified in Morris et al. (2020); Dwivedi et al. (2020) while comparing two models on a dataset on the similar range of model parameters.”
Parameter Prediction for Unseen Deep Architectures (2021, background): “[23] Vijay Prakash Dwivedi, Chaitanya K Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson.”
Rethinking Graph Transformers with Spectral Attention (2021, methodology): “When comparing to the state-of-the-art (SOTA) models in the literature in Figure 7, we observe that our SAN model consistently performs better on all synthetic datasets from [15], highlighting the strong expressive power of the model.”
Amalgamating Knowledge from Heterogeneous Graph Neural Networks (2021, background): “Graph neural networks (GNNs) have achieved unprecedented advances in recent years, showing promising performance in handling graph data lying in the non-Euclidean domain [16, 37, 53, 44, 6, 41, 19, 23, 36, 13, 18, 25, 42].”
Machine Learning on Graphs: A Model and Comprehensive Taxonomy (2020, background): “Recently, there has been progress in this direction and graph benchmarks with leaderboards have been proposed [35, 45, 55].”
Graph Clustering with Graph Neural Networks (2020, methodology): “This model has been used extensively to benchmark graph clustering methods [19], and has recently been used for experiments on state-of-the-art GNNs [16].”
Artificial Neural Networks and Machine Learning – ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part III (2021, methodology): “Increasing n for a Fixed Parameter Budget For our next experiment we examine the test performance of our models with increasing hyperparameter n, while constraining the parameter budget to approximately 100K and 400K parameters [9].”
Atomistic Line Graph Neural Network for improved materials property predictions (2021, background): “eij 1⁄4 e 1 ij þ SiLUðNormðAlhl 1 i þ Bh 1 j þ Ce 1 ij ÞÞ (4)”
Rethinking Graph Neural Architecture Search from Message-passing (2021, methodology): “Consequently, a new range of datasets across different real-world tasks is proposed in [5].”
Training Graph Neural Networks with 1000 Layers (2021, background): “GNNs have shown a lot of promise on recent large-scale graph datasets such as Benchmarking GNNs (Dwivedi et al., 2020), Open Graph Benchmark (OGB) (Hu et al.”
Combinatorial optimization and reasoning with graph neural networks (2021, background): “While there are some empirical studies (Dwivedi et al., 2020; You et al., 2020) investigating how different architectural design choices, e.”
PDFormer: Propagation Delay-aware Dynamic Long-range Transformer for Traffic Flow Prediction (2023, background): “Laplacian eigenvectors embed the graph in Euclidean space and preserve the global graph structure information (Dwivedi et al. 2020).”