Adversarial Graph Augmentation to Improve Graph Contrastive Learning

By Susheel Suresh, Pan Li, Cong Hao, Jennifer Neville. Neural Information Processing Systems, 2021

456 citations recorded by the source corpus.

Self-supervised learning of graph neural networks (GNN) is in great need because of the widespread label scarcity issue in real-world graph/network data. Graph contrastive learning (GCL), by training GNNs to maximize the correspondence between the representations of the same graph in its different augmented forms, may yield robust and transferable GNNs even without using labels. However, GNNs trained by traditional GCL often risk capturing redundant graph features and thus may be brittle and provide sub-par performance in downstream tasks. Here, we propose a novel principle, termed adversarial-GCL (AD-GCL), which enables GNNs to avoid capturing redundant information during the training by optimizing adversarial graph augmentation strategies used in GCL. We pair AD-GCL with theoretical explanations and design a practical instantiation based on trainable edge-dropping graph augmentation. We experimentally validate AD-GCL by comparing with the state-of-the-art GCL methods and achieve performance gains of up-to $14\%$ in unsupervised, $6\%$ in transfer, and $3\%$ in semi-supervised learning settings overall with 18 different benchmark datasets for the tasks of molecule property regression and classification, and social network classification.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/aa9ae8096216163ed40dd787917215b5ae4d3d90

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chuan Zhou
2. Philip S. Yu
3. Feng Xia
4. Shirui Pan
5. Yaochen Xie
6. Zhengyang Wang
7. Zhao Xu
8. Siqi Miao

In Computational Theory and Mathematics

1. Philip S. Yu
2. Pan Li
3. Shuiwang Ji
4. Junseok Lee
5. Chanyoung Park
6. Feng Xia
7. Shirui Pan
8. Namkyeong Lee

In Materials Chemistry

1. Philip S. Yu
2. Pan Li
3. Shuiwang Ji
4. Junseok Lee
5. Chanyoung Park
6. Siqi Miao
7. Miaoyuan Liu
8. Shirui Pan

Citations in the corpus

• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “[81] only considers parameterized structural transformations, and more complicated learnable view generation involving feature-space, structure-space, and sampling-based transformation are still challenging and are limited by the development of graph generation”
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, background): “Towards the line of contrastive SSL over graph structures, recent efforts focus on designing various graph contrastive learning methods (Yu et al., 2022b; Yin et al., 2022; Zhang et al., 2022; Xia et al., 2022a; Suresh et al., 2021).”
• Augmentation-Free Self-Supervised Learning on Graphs (2021, background): “…node dropping (You et al. 2020), edge modification (Jin et al. 2021; Qiu et al. 2020; Zhao et al. 2020), subgraph extraction (Jiao et al. 2020; Sun et al. 2021b), attribute masking (Zhu et al. 2020, 2021) and others (Hassani and Khasahmadi 2020; Kefato and Girdzijauskas 2021; Suresh et al. 2021).”
• Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism (2022, background): “Previous works have shown that label-irrelevant features are known to be discriminative enough to even identify each graph in the training dataset let alone the labels (Suresh et al., 2021).”
• Graph Self-Supervised Learning: A Survey (2021, background): “AD-GCL [75] proposes an adversarial edge dropping mechanism as augmentations to reduce the amount of redundant information”