Deep Graph Contrastive Representation Learning

By Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang. arXiv.org, 2020

1,103 citations recorded by the source corpus.

Graph representation learning nowadays becomes fundamental in analyzing graph-structured data. Inspired by recent success of contrastive methods, in this paper, we propose a novel framework for unsupervised graph representation learning by leveraging a contrastive objective at the node level. Specifically, we generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. To provide diverse node contexts for the contrastive objective, we propose a hybrid scheme for generating graph views on both structure and attribute levels. Besides, we provide theoretical justification behind our motivation from two perspectives, mutual information and the classical triplet loss. We perform empirical experiments on both transductive and inductive learning tasks using a variety of real-world datasets. Experimental experiments demonstrate that despite its simplicity, our proposed method consistently outperforms existing state-of-the-art methods by large margins. Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.

View Paper: https://www.semanticscholar.org/paper/4bf76588122827157c43a59e656dccc6b6a22e90

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Dacheng Tao
2. Lirong Wu
3. Yaochen Xie
4. Yixin Liu
5. Jie Gui
6. Jing Zhang
7. Shirui Pan
8. Ming-Hsuan Yang

Citations in the corpus

• A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, methodology): “It has been combined with clustering [66], semi-supervised learning [151], multi-task learning [163]–[166], transfer learning [167]–[169], graph NNs [153], [170]–[175], reinforcement learning [176]–[178], few-shot learning [179], [180], neural”
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, background): “In our model, † Due to space limit, results of NCF, GCCF, GraphCL, SAIL, GRACE, and AutoGCL are in Appendix B. ‡ In the subsequent experiments, we found that there is a better experimental setting that can produce a higher performance, so we updated the codes on Github to the new setting.”
• Mining Latent Structures for Multimedia Recommendation (2021, background): “GNNs have shown great power on analyzing graph-structured data and have been widely employed for graph analytical tasks across a variety of domains, including node classification [21, 50, 51], link prediction [45], information retrieval[44, 47], etc.”
• Graph Contrastive Learning with Adaptive Augmentation (2020, methodology): “gical and semantic graph patterns. 3.3.1 Topology-level augmentation. For topology level augmentation, we consider a direct way for corrupting input graphs where we randomly remove edges in the graph [51]. Formally, we sample a modified subsetEefrom the original Ewith probability �{(�,�)∈eE} = 1−���”
• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “GraphCL [49] and GRACE [46] include additional contrast among the same view of different instances (i.”
• From Canonical Correlation Analysis to Self-supervised Graph Neural Networks (2021, methodology): “Most of previous works highly rely on negative pairs to avoid collapse or interchangeable, trivial/degenerated solutions [48, 15, 57, 58].”
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, background): “After that, motivated by the great success of contrastive mechanisms [77], [78], [79], [80], contrastive learning models have been increasingly proposed, e.”
• Graph Contrastive Learning Automated (2021, background): “…representations from unlabeled graph data (Veli ˇ ckovi ´ c et al., 2018; Sun et al., 2019; You et al., 2020a; Qiu et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020b;c; Chen et al., 2020b;a; Ren et al., 2019; Park et al., 2020; Peng et al., 2020b; Jin et al., 2021a; Wang & Liu, 2021).”
• Augmentation-Free Self-Supervised Learning on Graphs (2021, methodology): “…on graphs, the quality of the learned graph representations of previous augmentation-based contrastive meth-ods (Hassani and Khasahmadi 2020; Zhu et al. 2020, 2021; Thakoor et al. 2021; Sun et al. 2021a; Veliˇckovi´c et al. 2018) is highly dependent on the choice of the augmentation scheme .”
• SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation (2022, background): “Typically, for node-level tasks [56, 57], GCA [58] argues that data augmentation schemes should preserve intrinsic structures and attributes of graphs and thus proposes to adopt adaptive augmentations that only perturb unimportant components.”
• Large-Scale Representation Learning on Graphs via Bootstrapping (2021, unknown): “…graph methods from previously published results (Perozzi et al., 2014; Veliˇckovi´c et al., 2019; Peng et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020a), as well as Random-Init (Veliˇckovi´c et al., 2019), a baseline using embeddings from a randomly initialized encoder, thus…”
• Bootstrapped Representation Learning on Graphs (2021, methodology): “In our experiments, we closely follow models and architectures as used in prior works (Zhu et al., 2020b; Veliˇckovi´c et al., 2019; Zhu et al., 2020a) for fair comparisons.”
• Diffusion Models: A Comprehensive Survey of Methods and Applications (2022, background): “Graph Neural Networks [85, 251, 266, 285] and corresponding representation learning [86] techniques have achieved great success [14, 231, 250, 258, 264, 288] in many areas, including modeling molecule graph in various tasks ranging from property prediction [59, 71] to molecule generation [105, 111, ”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, methodology): “GRACE [26].”
• Graph Self-Supervised Learning: A Survey (2021, methodology): “, column-wisely) mask a number of selected feature channels with zeros [33], [68].”
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning (2022, methodology): “For instance, GRACE [44] proposes a framework for node-level graph contrastive learning, and performs corruption by removing edges and masking node features.”