Graph Contrastive Learning with Augmentations

By Yuning You, Tianlong Chen, Yongduo Sui, Ting-Li Chen, Zhangyang Wang, Yang Shen. Neural Information Processing Systems, 2020

2,886 citations recorded by the source corpus.

Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at https://github.com/Shen-Lab/GraphCL.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/76c124786ccf4263e6403a15a8e350ac28be4e65

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Hongzhi Yin
4. Philip S. Yu
5. Jie Tang
6. Tong Chen
7. Chao Huang
8. Junliang Yu

In Information Systems

1. Hongzhi Yin
2. Xiangnan He
3. Xiang Wang
4. Tong Chen
5. Junliang Yu
6. Xin Xia
7. Chao Huang
8. Fuli Feng

Citations in the corpus

• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts (2023, neutral): “Selfsupervised learning is also popular in other fields like natural language processing [95], speech processing [96], and graph-structured data [97].”
• Pre-training Molecular Graph Representation with 3D Geometry (2021, methodology): “Contrastive graph SSL [38, 71, 79, 90, 91] constructs the supervised signals at the inter-graph level and learns the representation by contrasting with other graphs, while generative graph SSL [31, 38, 39, 47] focuses on reconstructing the original graph at the intra-graph level.”
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, methodology): “GraphCL (You et al., 2020) generates correlated graph representation views using various augmentation strategies, such as node/edge perturbation and attribute masking.”
• Protein Representation Learning by Geometric Structure Pretraining (2022, background): “…Hamilton et al., 2017), context prediction (Hu et al., 2019; Rong et al., 2020), node/edge attribute reconstruction (Hu et al., 2019) and contrastive learning (Hassani & Khasahmadi, 2020; Qiu et al., 2020; You et al., 2020; Xu et al., 2021), are designed to acquire knowledge from unlabeled graphs.”
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, methodology): “Recently, GraphCL [54] shows that GNNs can gain robustness using their proposed framework.”
• Deep Graph Clustering via Dual Correlation Reduction (2021, background): “Recent efforts in self-supervised graph representation learning have demonstrated that graph distortion could enable the network to learn rich representations from different contexts for nodes (Hassani and Khasahmadi 2020; You et al. 2020).”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, background): “GraphCL [25].”
• Graph Self-Supervised Learning: A Survey (2021, methodology): “For example, GraphCL(G) [38] adopts SimCLR [24] to form its contrastive pipeline which pulls the graph-level representations of two views closer.”
• Self-Supervised Learning: Generative or Contrastive (2020, background): “GraphCL [152] studies the data augmentation strategies in graph learning.”
• Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation (2021, background): “Unlike existing studies which enforce perturbations on graphs to augment the ground-truth [53], we propose to construct self-supervision signals by exploiting the hypergraph structures, with the intuition that the comprehensive user representation should reflect the user node’s local and global high”
• Self-supervised Graph Learning for Recommendation (2020, methodology): “SSL has also been applied on graph data [51, 52, 56].”
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, methodology): “We compare AD-GCL with three unsupervised/self-supervised learning baselines for graph-level tasks, which include randomly initialized untrained GIN (RU-GIN) [72], InfoGraph [18] and GraphCL [24].”
• Augmentation-Free Self-Supervised Learning on Graphs (2021, methodology): “Moreover, previous work (You et al. 2020; Sun et al. 2021a) have shown that there is no universally outperforming data augmentation scheme for graphs.”
• Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning (2022, background): “Contrastive Representation Learning Contrastive representation learning learns an embedding space where similar objects are closer than dissimilar ones, and has achieved great success in vision [7, 20, 6, 9], language [40, 16], and graph [51, 38].”
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction (2021, methodology): “Recently, more self-supervised tasks for GNNs have been proposed [14, 15, 52, 51, 32, 10, 54, 30].”
• Graph Contrastive Learning Automated (2021, methodology): “This section pass JOAO through a sanity check, by comparing its selections with the previous trial-and-error findings by manually and exhaustively combining different augmentations (using downstream labels for validation) (You et al., 2020a).”
• XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation (2022, methodology): “In this section, we investigate the widely used dropout-based augmentations on graphs [12], [4].”