Self-Supervised Learning of Graph Neural Networks: A Unified Review

By Yaochen Xie, Zhao Xu, Zhengyang Wang, Shuiwang Ji. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021

412 citations recorded by the source corpus.

Deep models trained in supervised mode have achieved remarkable success on a variety of tasks. When labeled samples are limited, self-supervised learning (SSL) is emerging as a new paradigm for making use of large amounts of unlabeled samples. SSL has achieved promising performance on natural language and image learning tasks. Recently, there is a trend to extend such success to graph data using graph neural networks (GNNs). In this survey, we provide a unified review of different ways of training GNNs using SSL. Specifically, we categorize SSL methods into contrastive and predictive models. In either category, we provide a unified framework for methods as well as how these methods differ in each component under the framework. Our unified treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms. We also summarize different SSL settings and the corresponding datasets used in each setting. To facilitate methodological development and empirical comparison, we develop a standardized testbed for SSL in GNNs, including implementations of common baseline methods, datasets, and evaluation metrics.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/8d68eae4068fca5ae3e9660c2a87857c89d30f73

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Shirui Pan
3. Hongzhi Yin
4. Tong Chen
5. Xin Xia
6. Chuan Zhou
7. Junliang Yu
8. Zi-Liang Huang

Citations in the corpus

• Pre-training Molecular Graph Representation with 3D Geometry (2021, methodology): “However, in the research line of graph SSL, as summarized in several recent survey papers [50, 84, 86], the two most used ones are InfoNCE and Jensen-Shannon Estimator (JSE) [36, 58].”
• Graph Contrastive Learning Automated (2021, background): “For comprehensive reviews of the topic, please refer to (Xie et al., 2021; Liu et al., 2021).”
• Self-supervised learning in medicine and healthcare (2022, background): “Molecules can also be augmented by randomly removing some bonds, or by removing whole portions of atoms and bonds from parts of the molecule 25 .”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, methodology): “advantages are as follow: (1) more mathematical details, striving to summarize each method with one equation; (2) more implementation details, including 41 datasets statistics (vs 20 datasets in [20]), evaluation metrics, and open-source code; (3) more thorough experimental study for fair comparison”
• Graph Self-Supervised Learning: A Survey (2021, background): “InfoNCE) provides a lower bound MI estimation that naturally consists of a positive and N negative pairs [36].”
• Self-Supervised Learning for Recommender Systems: A Survey (2022, background): “While SSL has been extensively surveyed in the fields of CV, NLP [34], [9] and graph learning [35], [36], [37], there has not been a systematic investigation of research endeavors on SSR despite the growing number of publications.”