Multi-Stage Self-Supervised Learning for Graph Convolutional Networks

By Ke Sun, Zhanxing Zhu, Zhouchen Lin. AAAI Conference on Artificial Intelligence, 2019

310 citations recorded by the source corpus.

Graph Convolutional Networks (GCNs) play a crucial role in graph learning tasks, however, learning graph embedding with few supervised signals is still a difficult problem. In this paper, we propose a novel training algorithm for Graph Convolutional Network, called Multi-Stage Self-Supervised (M3S) Training Algorithm, combined with self-supervised learning approach, focusing on improving the generalization performance of GCNs on graphs with few labeled nodes. Firstly, a Multi-Stage Training Framework is provided as the basis of M3S training method. Then we leverage DeepCluster technique, a popular form of self-supervised learning, and design corresponding aligning mechanism on the embedding space to refine the Multi-Stage Training Framework, resulting in M3S Training Algorithm. Finally, extensive experimental results verify the superior performance of our algorithm on graphs with few labeled nodes under different label rates compared with other state-of-the-art approaches.

Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/e49dca82bb4096dcd9a5560597f8ca64643b0090

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hongzhi Yin
2. Junliang Yu
3. Yaochen Xie
4. Zhengyang Wang
5. Zhao Xu
6. Jundong Li
7. Nguyen Quoc Viet Hung
8. Shuiwang Ji

In Information Systems

1. Hongzhi Yin
2. Junliang Yu
3. Jundong Li
4. Nguyen Quoc Viet Hung
5. Shuiwang Ji
6. Qinyong Wang
7. Xiangliang Zhang
8. Xin Xia

In Statistical and Nonlinear Physics

1. Yaochen Xie
2. Shuiwang Ji
3. Zhengyang Wang
4. Zhangyang Gao
5. Jundong Li
6. Cheng Tan
7. Zhao Xu
8. Stan.Z.Li

Citations in the corpus

• Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation (2021, background): “The latest advances in this area have extended self-supervised learning to graph representation learning [28, 29, 35, 37].”
• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “For node-level tasks, MVGRL [10] follows DGI to obtain negative samples by corrupting given graphs.”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, methodology): “In terms of predictive methods, the basic difference between methods is how to obtain pseudo-labels, and there are three main means: (1) numerical statistics, such as Node Property Prediction and PairwiseAttrSim [10]; (2) prediction results from the previous training stage, such as CAGNN [74] and M3”