Graph Self-Supervised Learning: A Survey
By Yixin Liu, Shirui Pan, Ming Jin, Chuan Zhou, Feng Xia, Philip S. Yu. 2021
758 citations recorded by the source corpus.
Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation (2022, background): “Recently, referring to CV in general, several innovations have been proposed [53], [54], reviewed in many surveys [24], [55], [56], [57], [58].”
- GraphGPT: Graph Instruction Tuning for Large Language Models (2023, background): “While there have been successful attempts to align LLMs with visual information, such as multimodal LLMs [23, 67], the alignment of LLMs with graph structures remains largely unexplored.”
- Pre-training Molecular Graph Representation with 3D Geometry (2021, neutral): “1, 2, 9, 17, 18, 19 [59] Yixin Liu, Shirui Pan, Ming Jin, Chuan Zhou, Feng Xia, and Philip S Yu.”
- Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning (2022, background): “Since the success of contrastive learning in CV [2, 14], contrastive learning has been widely applied on NLP [5], graph data mining [17, 34] and recommender systems [28, 37].”
- A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, background): “Differences from previous work : Previous works have provided reviews on SSL that cater to specific applications such as recommender systems [26], graphs [27], sequential transfer learning [28], videos [29], and adversarial pre-training of self-supervised deep networks [30].”
- A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection (2023, background): “A GNN learns node representations through two primary functions [40]: A GGREGATE ( · ) and C OMBINE ( · ) .”
- FedProto: Federated Prototype Learning across Heterogeneous Clients (2021, background): “Recently, selfsupervised learning strategies are incorporated into the local training phase to handle the heterogeneity challenges (Li, He, and Song 2021; Liu et al. 2021a).”
- Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “Recently, another concurrent survey [63] provides reviews on SSL methods for GNNs.”
- Graph Contrastive Learning Automated (2021, background): “For comprehensive reviews of the topic, please refer to (Xie et al., 2021; Liu et al., 2021).”