OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs

By Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, Jure Leskovec. 2021

508 citations recorded by the source corpus.

Enabling effective and efficient machine learning (ML) over large-scale graph data (e.g., graphs with billions of edges) can have a great impact on both industrial and scientific applications. However, existing efforts to advance large-scale graph ML have been largely limited by the lack of a suitable public benchmark. Here we present OGB Large-Scale Challenge (OGB-LSC), a collection of three real-world datasets for facilitating the advancements in large-scale graph ML. The OGB-LSC datasets are orders of magnitude larger than existing ones, covering three core graph learning tasks -- link prediction, graph regression, and node classification. Furthermore, we provide dedicated baseline experiments, scaling up expressive graph ML models to the massive datasets. We show that expressive models significantly outperform simple scalable baselines, indicating an opportunity for dedicated efforts to further improve graph ML at scale. Moreover, OGB-LSC datasets were deployed at ACM KDD Cup 2021 and attracted more than 500 team registrations globally, during which significant performance improvements were made by a variety of innovative techniques. We summarize the common techniques used by the winning solutions and highlight the current best practices in large-scale graph ML. Finally, we describe how we have updated the datasets after the KDD Cup to further facilitate research advances. The OGB-LSC datasets, baseline code, and all the information about the KDD Cup are available at https://ogb.stanford.edu/docs/lsc/ .

Subfields: Artificial Intelligence, Materials Chemistry, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Statistical and Nonlinear Physics

Citations in the corpus

Recipe for a General, Powerful, Scalable Graph Transformer (2022, methodology): “To ensure diverse benchmarking tasks, we use datasets from Benchmarking-GNNs [14], OGB [25] and its large-scale challenge [26], with more details given in Appendix A.”
GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks (2020, unknown): “GraphLIME bases on the following two points: (1) It has been shown that GNN model is usually nonlinear and deeper GNN models perform better than shallow models in GNN for extensive graph data [10], [11], thus using linear model to fit its decision process is unbefitting because of its nonlinearity a”
Do Transformers Really Perform Badly for Graph Representation? (2021, methodology): “By using the ensemble with ExpC [50], we got a 0.1200 MAE on complete test set and won the first place of the graph-level track in OGB Large-Scale Challenge[21, 52].”
Large-Scale Representation Learning on Graphs via Bootstrapping (2021, unknown): “OGB-LSC MAG240M Dataset: This is a heterogeneous graph introduced for the KDD Cup 2021 (Hu et al., 2021), comprised of 121 million academic papers, 122 million authors, and 26 thousand institutions.”
Long Range Graph Benchmark (2022, methodology): “The molecules are treated as relational graphs with learnable edge types (standard BondEncoder in OGB [23] e.g., single bond, double bond, triple bond), but the predictable contact link does not have an explicit edge type, so a link prediction decoder is a function f(h, t) of probed head and tail no”
Do Transformers Really Perform Bad for Graph Representation? (2021, methodology): “In this paper, we give an affirmative answer by developing Graphormer, which is directly built upon the standard Transformer, and achieves state-of-the-art performance on a wide range of graphlevel prediction tasks, including the very recent Open Graph Benchmark Large-Scale Challenge (OGB-LSC) [20],”
Benchmarking Graph Neural Networks (2023, background): “Finally, very large graph datasets also present a computational challenge and require extensive GPU resources to be studied (Chiang et al., 2019; Rossi et al., 2020; Hu et al., 2021).”
Discovering Invariant Rationales for Graph Neural Networks (2022, background): “• Molhiv (OGBG-Molhiv) (Hu et al., 2020; 2021; Wu et al., 2017) is a molecular property prediction dataset consisting of molecule graphs, where nodes are atoms, and edges are chemical bonds.”
Training Graph Neural Networks with 1000 Layers (2021, background): “We also anticipate that overparameterized GNNs will be a promising solution to even larger datasets such as OGB-LSC (Hu et al., 2021).”