Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive

By Lirong Wu, Zhangyang Gao, Cheng Tan, Stan.Z.Li. IEEE Transactions on Knowledge and Data Engineering, 2021

336 citations recorded by the source corpus.

Deep learning on graphs has recently achieved remarkable success on a variety of tasks, while such success relies heavily on the massive and carefully labeled data. However, precise annotations are generally very expensive and time-consuming. To address this problem, self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks without relying on manual labels. In this survey, we extend the concept of SSL, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive. More importantly, unlike other surveys that only provide a high-level description of published research, we present an additional mathematical summary of existing works in a unified framework. Furthermore, to facilitate methodological development and empirical comparisons, we also summarize the commonly used datasets, evaluation metrics, downstream tasks, open-source implementations, and experimental study of various algorithms. Finally, we discuss the technical challenges and potential future directions for improving graph self-supervised learning. Latest advances in graph SSL are summarized in a GitHub repository https://github.com/LirongWu/awesome-graph-self-supervised-learning.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2d00798b8a7d979c925901e9faa5fe4360030ca2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jing Zhang
2. Dacheng Tao
3. Jie Gui
4. Zhe Sun
5. Qiong Cao
6. Tuo Chen
7. Haoran Luo
8. Bo Du

Citations in the corpus

• A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, background): “Differences from previous work: Previous works have provided reviews on SSL that cater to specific applications such as recommender systems [26], graphs [27], [28], sequential transfer learning [29], videos [30], adversarial pretraining of self-supervised deep networks [31], and visual feature learn”