Graph Representation Learning via Graphical Mutual Information Maximization

By Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, Junzhou Huang. The Web Conference, 2020

693 citations recorded by the source corpus.

The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision. This paper investigates how to preserve and extract the abundant information from graph-structured data into embedding space in an unsupervised manner. To this end, we propose a novel concept, Graphical Mutual Information (GMI), to measure the correlation between input graphs and high-level hidden representations. GMI generalizes the idea of conventional mutual information computations from vector space to the graph domain where measuring mutual information from two aspects of node features and topological structure is indispensable. GMI exhibits several benefits: First, it is invariant to the isomorphic transformation of input graphs—an inevitable constraint in many existing graph representation learning algorithms; Besides, it can be efficiently estimated and maximized by current mutual information estimation methods such as MINE; Finally, our theoretical analysis confirms its correctness and rationality. With the aid of GMI, we develop an unsupervised learning model trained by maximizing GMI between the input and output of a graph neural encoder. Considerable experiments on transductive as well as inductive node classification and link prediction demonstrate that our method outperforms state-of-the-art unsupervised counterparts, and even sometimes exceeds the performance of supervised ones.

Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/896f2aac3d9f32cd676442de2f88e8a802d8c349

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Chuan Shi
3. Xiao Wang
4. Hongzhi Yin
5. Shirui Pan
6. Wayne Xin Zhao
7. Pan Li
8. Shuiwang Ji

In Information Systems

1. Hongzhi Yin
2. Philip S. Yu
3. Wayne Xin Zhao
4. Junliang Yu
5. Chuan Shi
6. Chao Huang
7. Lianghao Xia
8. Yupeng Hou

In Statistical and Nonlinear Physics

1. Xiao Wang
2. Chuan Shi
3. Philip S. Yu
4. Shirui Pan
5. Chuan Zhou
6. Pan Li
7. Yixin Liu
8. Shuiwang Ji

Citations in the corpus

• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation (2023, background): “Recently, self-supervised learning (SSL) has advanced the graph learning paradigm by enhancing node representation from unlabeled graph data (Zhu et al., 2021a;b; Velickovic et al., 2019; Hassani & Khasahmadi, 2020; Peng et al., 2020; Zhu et al., 2020; Wu et al., 2022).”
• Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive (2021, background): “cess [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], it is still very challenging to deal with the inherent differences between grid-like and structured-like data.”
• Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, methodology): “[42] derives the graphic mutual information (GMI) based on MI decomposition and optimizes GMI via JS estimator.”
• From Canonical Correlation Analysis to Self-supervised Graph Neural Networks (2021, background): “[31] Zhen Peng, Wenbing Huang, Minnan Luo, Qinghua Zheng, Yu Rong, Tingyang Xu, and Junzhou Huang.”
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning (2022, background): “Besides, some works [21, 34, 44] make the contrast between the learned node representations and the input node features, while we make the contrast with representations of homogeneous neighbors, which is more suited to the recommendation task.”
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning (2021, background): “GMI [58] considers maximizing the agreement between node’s embedding and raw features of its neighbors as well as which between embedding of two adjacent nodes.”
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction (2021, positive): “Furthermore, [39, 28, 41] proposed to use the mutual information maximization as the optimization objective for GNNs.”
• Augmentation-Free Self-Supervised Learning on Graphs (2021, methodology): “No Augmentation No Negative Sampling DGI GMI MVGRL GRACE GCA BGRL AFGRL Table 6: Properties of methods that are compared.”
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning (2021, background): “Along this line, GMI [25] is proposed to contrast between center node and its local patch from node features and topological structure.”
• Bootstrapped Representation Learning on Graphs (2021, methodology): “GMI (Peng et al., 2020) directly maximizes a notion of graphical mutual information inspired by MINE (Belghazi et al.”
• Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation (2021, background): “The latest advances in this area have extended self-supervised learning to graph representation learning [28, 29, 35, 37].”
• Graph Contrastive Learning with Adaptive Augmentation (2020, methodology): “GI, previous work [51] proposes to not rely on an explicit graph embedding, but rather focus on maximizing the agreement of node embeddings across two corrupted views of the graph. Following DGI, GMI [30] employs two discriminators to directly measure MI between input and representations of both nod”
• Self-Supervised Graph Transformer on Large-Scale Molecular Data (2020, background): “[55, 36, 50] exploit the mutual information maximization scheme to construct objective for GNNs. Recently, two works are proposed to construct unsupervised representations for molecular graphs.”
• Graph Information Bottleneck (2020, positive): “…as a mutual information maximization term that aims to maximize the mutual information between representations of nodes and their neighbor patches [37], between representations of sub-structures and the hidden feature vectors [38], between representations of graphs and their sub-structures [39].”
• Graph Contrastive Learning Automated (2021, background): “…representations from unlabeled graph data (Veli ˇ ckovi ´ c et al., 2018; Sun et al., 2019; You et al., 2020a; Qiu et al., 2020; Hassani & Khasahmadi, 2020; Zhu et al., 2020b;c; Chen et al., 2020b;a; Ren et al., 2019; Park et al., 2020; Peng et al., 2020b; Jin et al., 2021a; Wang & Liu, 2021).”