Heterogeneous Graph Neural Network

By Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, Nitesh V. Chawla. Knowledge Discovery and Data Mining, 2019

1,630 citations recorded by the source corpus.

Representation learning in heterogeneous graphs aims to pursue a meaningful vector representation for each node so as to facilitate downstream applications such as link prediction, personalized recommendation, node classification, etc. This task, however, is challenging not only because of the demand to incorporate heterogeneous structural (graph) information consisting of multiple types of nodes and edges, but also due to the need for considering heterogeneous attributes or contents (e.g., text or image) associated with each node. Despite a substantial amount of effort has been made to homogeneous (or heterogeneous) graph embedding, attributed graph embedding as well as graph neural networks, few of them can jointly consider heterogeneous structural (graph) information as well as heterogeneous contents information of each node effectively. In this paper, we propose HetGNN, a heterogeneous graph neural network model, to resolve this issue. Specifically, we first introduce a random walk with restart strategy to sample a fixed size of strongly correlated heterogeneous neighbors for each node and group them based upon node types. Next, we design a neural network architecture with two modules to aggregate feature information of those sampled neighboring nodes. The first module encodes "deep" feature interactions of heterogeneous contents and generates content embedding for each node. The second module aggregates content (attribute) embeddings of different neighboring groups (types) and further combines them by considering the impacts of different groups to obtain the ultimate node embedding. Finally, we leverage a graph context loss and a mini-batch gradient descent procedure to train the model in an end-to-end manner. Extensive experiments on several datasets demonstrate that HetGNN can outperform state-of-the-art baselines in various graph mining tasks, i.e., link prediction, recommendation, node classification & clustering and inductive node classification & clustering.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/e3d662bbd0e5539fe22a85f3518f960595b9914e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chuan Shi
2. Jie Tang
3. Yuxiao Dong
4. Xiao Wang
5. Yizhou Sun
6. Quanming Yao
7. Tong Zhang
8. Xiangnan He

In Statistical and Nonlinear Physics

1. Chuan Shi
2. Yuxiao Dong
3. Xiao Wang
4. Jie Tang
5. Kuansan Wang
6. Quanming Yao
7. Binbin Hu
8. Tong Zhang

Citations in the corpus

• Heterogeneous Graph Transformer (2020, methodology): “Moreover, HGT has fewer parameters and comparable batch time than all the heterogeneous graph neural network baselines, including RGCN, HetGNN, and HAN.”
• Continual Graph Learning (2020, neutral): “Recently, a wide range of graph neural networks (GNNs) have been proposed to exploit the structural information underlying graphs, to potentially benefit a variety of applications [Kipf and Welling, 2017; Gao et al. , 2018; Wu et al. , 2019a; Zhang et al. , 2019].”
• Graph Few-shot Learning via Knowledge Transfer (2019, background): “Graph Neural Network Recently, a variety of graph neural network models (GNN) have been proposed to exploit the structures underlying graphs to benefit a variety of applications (Kipf and Welling 2017; Zhang et al. 2019; Tang et al. 2019; Huang et al. 2019; Liu et al. 2019; Gao, Wang, and Ji 2018).”
• A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources (2020, neutral): “HetGNN [16] is the representative work of unsupervised HGNNs.”
• Graph Transformer Networks (2019, background): “Also, as several heterogeneous graph datasets have been recently studied for other network analysis tasks, such as link prediction [36, 41] and graph classification [17, 24], applying our GTNs to the other tasks can be interesting future directions.”
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “• HetGNN [229].”
• DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks (2020, background): “More recently, some works have extended NAS to homogeneous GNNs [8, 16, 33, 42].”
• A Comprehensive Survey on Graph Anomaly Detection With Deep Learning (2021, background): “Opportunities: Although plenty of deep learning techniques have been developed to analyze heterogeneous graphs [151], [152], incorporating the most recent techniques for graph anomaly detection is uncharted yet.”
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning (2021, methodology): “Baselines We compare the proposed HeCo with three categories of baselines: unsupervised homogeneous methods { GraphSAGE [10], GAE [18], DGI [30] }, unsupervised heterogeneous methods { Mp2vec [4], HERec [26], HetGNN [34], DMGI [24] }, and a semisupervised heterogeneous method HAN [31].”
• Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay (2021, background): “They also potentially bene-fit many real-world applications, ranging from node classi-fication and link prediction to traffic prediction and better recommendation (Kipf and Welling 2017; Gao, Wang, and Ji 2018; Wu et al. 2019a; Zhang et al. 2019).”
• Graph Neural Networks and Their Current Applications in Bioinformatics (2021, background): “…current GNNs mainly focus on the processing of isomorphic graphs and cannot sufficiently capture the heterogeneity of nodes and edges in Li C. et al., 2019; Zhang J. et al., 2019; Ding et al., 2020; Li J. et al., 2020; Wu et al., 2020; Zheng et al., heterogeneous networks (Zhang C. et al., 2019).”
• Heterogeneous Graph Structure Learning for Graph Neural Networks (2021, background): “Most HGNNs follow a message-passing scheme where the node embedding is learned by aggregating and transforming the embeddings of its original neighbors (Zhang et al. 2019a; Hu et al. 2020; Zhao et al. 2020a; Hong et al. 2020) or metapath-based neighbors (Wang et al.”
• Are we really making much progress?: Revisiting, benchmarking and refining heterogeneous graph neural networks (2021, methodology): “HAN [36] GTN [43] RSHN [45] HetGNN [44] MAGNN [12] Dataset The main drawback of GTN is that it consumes gigantic amount of time and memory.”
• Data Augmentation for Graph Neural Networks (2020, methodology): “At the same time, graph neural networks (GNNs) (Hamilton, Ying, and Leskovec 2017; Kipf and Welling 2016a; Veličković et al. 2017; Xu et al. 2018a; Zhang et al. 2019a; Chen, Ma, and Xiao 2018; Wu et al. 2019; Zhang, Cui, and Zhu 2018; Xu et al. 2018b) have emerged as a rising approach for…”
• CGLB: Benchmark Tasks for Continual Graph Learning (2022, background): “Several representative heterogeneous GNNs including Heterogeneous Graph Neural Network (HetGNN) [62], Heterogeneous Graph Attention Network (HAN) [49], Heterogeneous Graph Transformer (HGT) [22], Relational Graph Convolutional Networks (R-GCNs) [41], Metapath2vec [11], and Predictive Text Embedding…”