Heterogeneous Graph Attention Network

By Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Peng Cui, Philip S. Yu, Yanfang Ye. The Web Conference, 2019

3,241 citations recorded by the source corpus.

Graph neural network, as a powerful graph representation technique based on deep learning, has shown superior performance and attracted considerable research interest. However, it has not been fully considered in graph neural network for heterogeneous graph which contains different types of nodes and links. The heterogeneity and rich semantic information bring great challenges for designing a graph neural network for heterogeneous graph. Recently, one of the most exciting advancements in deep learning is the attention mechanism, whose great potential has been well demonstrated in various areas. In this paper, we first propose a novel heterogeneous graph neural network based on the hierarchical attention, including node-level and semantic-level attentions. Specifically, the node-level attention aims to learn the importance between a node and its meta-path based neighbors, while the semantic-level attention is able to learn the importance of different meta-paths. With the learned importance from both node-level and semantic-level attention, the importance of node and meta-path can be fully considered. Then the proposed model can generate node embedding by aggregating features from meta-path based neighbors in a hierarchical manner. Extensive experimental results on three real-world heterogeneous graphs not only show the superior performance of our proposed model over the state-of-the-arts, but also demonstrate its potentially good interpretability for graph analysis.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/00b7efbf14a54cced4b9f19e663b70ffbd01324b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Shirui Pan
3. Zhiyuan Liu
4. Chuan Shi
5. Xiao Wang
6. Maosong Sun
7. Chengqi Zhang
8. Jie Tang

In Statistical and Nonlinear Physics

1. Chuan Shi
2. Xiao Wang
3. Philip S. Yu
4. Shirui Pan
5. Chengqi Zhang
6. Yuxiao Dong
7. Quanming Yao
8. Guodong Long

Citations in the corpus

• Implicit Graph Neural Networks (2020, methodology): “Consistent with previous works (Park et al., 2019), we use the the publicly available ACM data set (Wang et al., 2019), preprocessed DBLP and IMDB data sets (Park et al., 2019).”
• Deep Learning with Graph Convolutional Networks: An Overview and Latest Applications in Computational Intelligence (2023, background): “Te key points of this survey include the following points: (i) Te existing applications in the feld of computer mainly use GN, GCN, and MPNN models, rarely use the GAE model and do not use the GATmodel.”
• Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay (2021, methodology): “Spatial methods aggregate the node representations directly from its neighborhood (Hamilton, Ying, and Leskovec 2017; Veliˇckovi´c et al. 2018; Wang et al. 2019), while the basic idea behind spectral approaches is to learn graph representation in the spectral domain where the learned filters are…”
• Global Context Enhanced Graph Neural Networks for Session-based Recommendation (2020, methodology): “To prevent GCE-GNN from overfitting, we employ dropout [14] regularization techniques, which have been shown to be effective in various neural network architectures including graph neural networks[22][23].”
• LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding (2020, methodology): “Referring to the exiting works (Jiao et al. 2019; Wang et al. 2019b), we use matrix W ∈ Rd×d′ to map the outputs of different dimensions to the same feature space.”
• Learning Graph Embedding With Adversarial Training Methods (2019, background): “The fundamental idea is to preserve the topological structure, vertex content, and other side information [10], [11].”
• Deep Learning on Graphs: A Survey (2018, neutral): “HAN [59] proposed a two-level attention mechanism, i.e., a node-level and a semantic-level attention mechanism, for heterogeneous graphs.”
• DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks (2020, methodology): “The dimension of semantic-level attention vector for HAN [26] and MAGNN [7] is 128.”
• Graph Transformer Networks (2019, methodology): “Previous works [37, 43] require manually defined meta-paths and perform Graph Neural Networks on the meta-path graphs.”
• Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering (2020, background): “[ Wang et al. , 2019b; Hu et al. , 2019a] propose heterogeneous graph attention networks with dual-level attention mechanism.”
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning (2021, methodology): “Moreover, even HAN utilizes the label information, HeCo performs better than it in all cases.”
• Structural Deep Clustering Network (2020, background): “• ACM2[28]: This is a paper network from the ACM dataset.”
• GPT-GNN: Generative Pre-Training of Graph Neural Networks (2020, methodology): “Therefore, in addition to HGT [15], we try GCN [17], GAT [36], RGCN [28], and HAN [39] as the base model.”
• A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources (2020, neutral): “HAN [15] designs a TABLE 2 Link Prediction of Dynamic HGE Methods”
• CGLB: Benchmark Tasks for Continual Graph Learning (2022, background): “Several representative heterogeneous GNNs including Heterogeneous Graph Neural Network (HetGNN) [62], Heterogeneous Graph Attention Network (HAN) [49], Heterogeneous Graph Transformer (HGT) [22], Relational Graph Convolutional Networks (R-GCNs) [41], Metapath2vec [11], and Predictive Text Embedding…”