Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning

By Xiao Wang, Nian Liu, Hui-jun Han, Chuan Shi. Knowledge Discovery and Data Mining, 2021

542 citations recorded by the source corpus.

Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use in reality since labels are usually scarce in real applications. Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo. Different from traditional contrastive learning which only focuses on contrasting positive and negative samples, HeCo employs cross-view contrastive mechanism. Specifically, two views of a HIN (network schema and meta-path views) are proposed to learn node embeddings, so as to capture both of local and high-order structures simultaneously. Then the cross-view contrastive learning, as well as a view mask mechanism, is proposed, which is able to extract the positive and negative embeddings from two views. This enables the two views to collaboratively supervise each other and finally learn high-level node embeddings. Moreover, two extensions of HeCo are designed to generate harder negative samples with high quality, which further boosts the performance of HeCo. Extensive experiments conducted on a variety of real-world networks show the superior performance of the proposed methods over the state-of-the-arts.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/e70930b8a8c7ba7af5f3df13ecf7b60364a4c189

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chuan Zhou
2. Philip S. Yu
3. Feng Xia
4. Shirui Pan
5. Yaochen Xie
6. Zhengyang Wang
7. Zhao Xu
8. Shuiwang Ji

In Information Systems

1. Shirui Pan
2. Jia Wu
3. Philip S. Yu
4. Cécile Paris
5. Surya Nepal
6. Quan Z. Sheng
7. Shuiwang Ji
8. Chuan Zhou

Citations in the corpus

• Graph Self-Supervised Learning: A Survey (2021, background): “Apart from those methods relying on carefully-crafted negative samples, approaches like BGRL [84] propose to contrast on graph instances themselves and thus alleviate TABLE 3 Main Characteristics of Contrast-Based Graph SSL Approaches Approach Pretext Task Category Downstream Task Level Training Sch”
• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, background): “In graph data analysis, SSL can potentially be of great importance to make use of a massive amount of unlabeled graphs such as molecular graphs [21, 22].”
• A Comprehensive Survey on Community Detection With Deep Learning (2021, methodology): “In another example, HeCo [100] exploits an attention mechanism [97] to embed the network schema and meta paths for preserving local and high-order structures.”
• A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources (2020, unknown): “HeCo [74] and HDGI [75] uses self-supervised learning, i.”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, background): “36, SelfGNN defines the implicit contrastive term directly in the form of MSE, HeCo [47].”