Open Graph Benchmark: Datasets for Machine Learning on Graphs

By Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec. 2020

3,715 citations recorded by the source corpus.

We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale (up to 100+ million nodes and 1+ billion edges), encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scripts, baseline code, and leaderboards are publicly available at this https URL .

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Masked Label Prediction: Unified Massage Passing Model for Semi-Supervised Classification (2020, methodology): “Experimental results show that UniMP outperforms the previous state-of-the-art models on three main OGBN datasets: ogbn-products , ogbn-proteins and ogbn-arxiv by a large margin, and ablation studies demonstrate the effectiveness of unifying feature propagation and label propagation.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “Multi-lingual models and benchmarks (e.g., Conneau et al. (2018); Conneau and Lample (2019); Hu et al. (2020); Clark et al. (2020)) represent another source of subpopulation shifts with corresponding disparities in model performance: training sets might contain fewer examples in low-resource…”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, background): “Accordingly, various benchmark datasets following diferent distributional shifts have been curated [54, 72, 89, 191].”
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021, background): “[52] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele 412 Catasta, and Jure Leskovec.”
Secure Deep Graph Generation with Link Differential Privacy (2020, background): “Nowadays, open data of networks (graphs) play a pivotal role in data mining and data analytics [Hu et al., 2020; Xie et al., 2020].”
Machine Learning on Graphs: A Model and Comprehensive Taxonomy (2020, background): “Recently, there has been progress in this direction and graph benchmarks with leaderboards have been proposed [35, 45, 55].”
Can graph neural networks count substructures? (2020, unknown): “†: Reported on the OGB learderboard [20].”
TUDataset: A collection of benchmark datasets for learning with graphs (2020, background): “Recently, ogb.stanford.edu (Hu et al., 2020) launched, however, the provided datasets for graph classiﬁcation focus on chemistry and bioinformatic applications, and the number is quite limited at this point.”
Graph Neural Networks with Heterophily (2020, methodology): “We assign to the nodes feature vectors from the recently announced Open Graph Benchmark (Hu et al. 2020), which includes only graphs with homophily.”
Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks (2020, background): “Due to the issues with existing datasets, there has been signiﬁcant concurrent work on establishing robust GNN benchmarks: • The Open Graph Benchmark (Hu et al., 2020) (OGB) has recently developed a range of datasets, focusing on diversity of domains, graph sizes and types of tasks and uniﬁed evalua”
Rethinking Graph Transformers with Spectral Attention (2021, methodology): “Figure 5: Effect of the γ parameter on the performance across datasets from [15, 21], using the Node LPE.”
Recipe for a General, Powerful, Scalable Graph Transformer (2022, methodology): “Table 4: Test performance in graph-level OGB benchmarks [25].”
Language is All a Graph Needs (2023, methodology): “Specifically, we select the following three popular citation graphs: ogbn-arxiv (Hu et al., 2020), Cora and PubMed (Yang et al., 2016), in which every node represents an academic paper on a specific topic, with its title and abstract included in raw text format.”
DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs (2020, methodology): “We benchmark the state-of-the-art GraphSAGE [2] model on two Open Graph Benchmark (OGB) datasets [33] shown in Table I.”
Large-Scale Representation Learning on Graphs via Bootstrapping (2021, unknown): “To study the tradeoff between performance and complexity we consider a node classiﬁcation task on a much larger dataset, from the OGB benchmark (Hu et al., 2020a), ogbn-arXiv.”
Large language models empowered agent-based modeling and simulation: a survey and perspectives (2023, unknown): “Landmark benchmarks like ImageNet (Russakovsky et al., 2015), GLUE (Wang et al., 2019), and the benchmarks in graph learning (Dwivedi et al., 2023; Hu et al., 2020) have been pivotal to the rapid innovation in the ﬁ elds of computer vision, natural language processing, and graph neural networks.”