SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation

By Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, Stan Z. Li. The Web Conference, 2022

452 citations recorded by the source corpus.

Graph contrastive learning (GCL) has emerged as a dominant technique for graph representation learning which maximizes the mutual information between paired graph augmentations that share the same semantics. Unfortunately, it is difficult to preserve semantics well during augmentations in view of the diverse nature of graph data. Currently, data augmentations in GCL broadly fall into three unsatisfactory ways. First, the augmentations can be manually picked per dataset by trial-and-errors. Second, the augmentations can be selected via cumbersome search. Third, the augmentations can be obtained with expensive domain knowledge as guidance. All of these limit the efficiency and more general applicability of existing GCL methods. To circumvent these crucial issues, we propose a Simple framework for GRAph Contrastive lEarning, SimGRACE for brevity, which does not require data augmentations. Specifically, we take original graph as input and GNN model with its perturbed version as two encoders to obtain two correlated views for contrast. SimGRACE is inspired by the observation that graph data can preserve their semantics well during encoder perturbations while not requiring manual trial-and-errors, cumbersome search or expensive domain knowledge for augmentations selection. Also, we explain why SimGRACE can succeed. Furthermore, we devise adversarial training scheme, dubbed AT-SimGRACE, to enhance the robustness of graph contrastive learning and theoretically explain the reasons. Albeit simple, we show that SimGRACE can yield competitive or better performance compared with state-of-the-art methods in terms of generalizability, transferability and robustness, while enjoying unprecedented degree of flexibility and efficiency. The code is available at: https://github.com/junxia97/SimGRACE.

Subfields: Artificial Intelligence, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/1fd4b505743f03f9f35327b219d28231a0c3796c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lianghao Xia
2. Chao Huang
3. Xubin Ren
4. Xuheng Cai

In Management Science and Operations Research

1. Lianghao Xia
2. Chao Huang
3. Xubin Ren
4. Xuheng Cai

Citations in the corpus

• GraphGPT: Graph Instruction Tuning for Large Language Models (2023, unknown): “The results, presented in Table 5 and 10 in the appendix, clearly demonstrate that ChatGPT, despite its massive parameter count (over 200B), struggles to make accurate predictions based solely on node text information or node content with the text-based graph structure.”
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, methodology): “• SimGRACE (Xia et al., 2022a) creates augmented view by randomly perturbing the parameters of the GNN network.”