Graph Contrastive Learning with Adaptive Augmentation

By Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, Liang Wang. The Web Conference, 2020

1,475 citations recorded by the source corpus.

Recently, contrastive learning (CL) has emerged as a successful method for unsupervised graph representation learning. Most graph CL methods first perform stochastic augmentation on the input graph to obtain two graph views and maximize the agreement of representations in the two views. Despite the prosperous development of graph CL methods, the design of graph augmentation schemes—a crucial component in CL—remains rarely explored. We argue that the data augmentation schemes should preserve intrinsic structures and attributes of graphs, which will force the model to learn representations that are insensitive to perturbation on unimportant nodes and edges. However, most existing methods adopt uniform data augmentation schemes, like uniformly dropping edges and uniformly shuffling features, leading to suboptimal performance. In this paper, we propose a novel graph contrastive representation learning method with adaptive augmentation that incorporates various priors for topological and semantic aspects of the graph. Specifically, on the topology level, we design augmentation schemes based on node centrality measures to highlight important connective structures. On the node attribute level, we corrupt node features by adding more noise to unimportant node features, to enforce the model to recognize underlying semantic information. We perform extensive experiments of node classification on a variety of real-world datasets. Experimental results demonstrate that our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts, which validates the effectiveness of the proposed contrastive framework with adaptive augmentation.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/0d67d3ddca1c4e370eaf1e99ec674f612c39c66c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiangnan He
2. Xiang Wang
3. Wayne Xin Zhao
4. Chao Huang
5. Fuli Feng
6. Lianghao Xia
7. Yupeng Hou
8. Dawei Yin

In Information Systems

1. Xiangnan He
2. Xiang Wang
3. Wayne Xin Zhao
4. Fuli Feng
5. Chao Huang
6. Lianghao Xia
7. Yupeng Hou
8. Dawei Yin

Citations in the corpus

• Mining Latent Structures for Multimedia Recommendation (2021, background): “GNNs have shown great power on analyzing graph-structured data and have been widely employed for graph analytical tasks across a variety of domains, including node classification [21, 50, 51], link prediction [45], information retrieval[44, 47], etc.”
• Hypergraph Contrastive Collaborative Filtering (2022, background): “It has achieved promising performance in various domains, such as visual data representation [5, 28], language data understanding [2, 31], graph representation learning [29, 51] and recommender systems [22, 38, 45, 46].”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, methodology): “GCA [40] adopts a similar framework to GRACE, but focuses on designing the adaptive augmentation strategy.”
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2021, background): “They either leverage domain knowledge [25,28,30], e.g. , node centralities in network science or molecule motifs in bio-chemistry, or depend on extensive evaluation on the downstream tasks, where the best strategy is selected based on validation performance [24,30].”
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning (2022, background): “As for graph contrastive learning, existing studies can be categorized into node-level contrastive learning [29, 45] and graph-level contrastive learning [26, 41].”
• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “Centrality-based edge removal [45] randomly removes edges based on pre-computed probabilities determined by the centrality score of each edge.”
• Large-Scale Representation Learning on Graphs via Bootstrapping (2021, neutral): “…graph methods from previously published results (Perozzi et al., 2014; Veliˇckovi´c et al., 2019; Peng et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020a), as well as Random-Init (Veliˇckovi´c et al., 2019), a baseline using embeddings from a randomly initialized encoder, thus…”
• Graph Contrastive Learning Automated (2021, background): “…representations from unlabeled graph data (Veli ˇ ckovi ´ c et al., 2018; Sun et al., 2019; You et al., 2020a; Qiu et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020b;c; Chen et al., 2020b;a; Ren et al., 2019; Park et al., 2020; Peng et al., 2020b; Jin et al., 2021a; Wang & Liu, 2021).”
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, methodology): “• GCA (Zhu et al., 2021b) adaptively dropout the nodes and edges by their importance calculated with node centrality.”
• Self-supervised Graph Learning for Recommendation (2020, methodology): “SSL has also been applied on graph data [51, 52, 56].”
• GraphGPT: Graph Instruction Tuning for Large Language Models (2023, neutral): “This difficulty is particularly pronounced when dealing with papers that have a high degree of cross-disciplinary characteristics, as illustrated in the example of machine learning and hardware architecture.”
• Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction (2022, methodology): “Inspired by the data augmentation strategy in (Zhu et al. 2021), we design an augmentation operator over the constructed traffic tensor X t − T : t , which is adaptive to the learned time-aware traffic pattern dependencies of each region.”