GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs

By Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, Dit-Yan Yeung. 2018

641 citations recorded by the source corpus.

We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head's importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Graph Infomax (2018, methodology): “We also provide supervised results for two related architectures— FastGCN (Chen et al., 2018) and Avg. pooling (Zhang et al., 2018).”
Masked Label Prediction: Unified Massage Passing Model for Semi-Supervised Classification (2020, methodology): “, 2017], GAAN [Zhang et al., 2018], our training strategy can be easily extended in kinds of GNNs such as GCN and GAT to further improve their performance.”
Simplifying Graph Convolutional Networks (2019, methodology): “We replace the GCN in DCGCN (Zhang et al., 2018b) with an SGC and get 71.0% and 76.2% on NCI1 and COLLAB datasets (Yanardag & Vishwanathan, 2015) respectively, which is on par with an GCN counterpart, but far behind GIN.”
Simple and Deep Graph Convolutional Networks (2020, methodology): “We compare GCNII with the following state-of-the-art methods: GraphSAGE (Hamilton et al., 2017), VR-GCN (Chen et al., 2018b), GaAN (Zhang et al., 2018), GAT (Veličković et al., 2018), JKNet (Xu et al., 2018), GeniePath (Liu et al., 2019), Cluster-GCN (Chiang et al., 2019).”
Graph Representation Learning via Graphical Mutual Information Maximization (2020, methodology): “We cite the results of DGI, GAT, FastGCN [7], and GaAN [48] in their original”
The Klarna Product Page Dataset: A Realistic Benchmark for Web Representation Learning (2021, background): “Over the recent years, there has been an upsurge in papers about node classiﬁcation on learning low-dimensional representations with GCNs. GCNs have shown notable performance on social network graphs [39, 6, 5], citation networks [43, 9], and biochemical graphs [6, 39, 24].”
Robust Graph Convolutional Networks Against Adversarial Attacks (2019, background): “Further improvements include adding an attention mechanism to assign different weights in aggregating node neighborhoods [28, 30, 31, 34], adding residual and jumping connections [32], sampling to improve efficiency [4, 5, 14, 33] considering edge attributes [15, 23, 26], disentangling node represen”
CoCoGUM: Contextual Code Summarization with Multi-Relational GNN on UMLs (2020, background): “Due to the strong ability of learning from graph data, GNNs have achieve a great success in many tasks like learning chemistry rule and phenomenon [31], recommender systems [32], document classification [33], traffic predication [34], etc.”
Combining Graph Neural Networks With Expert Knowledge for Smart Contract Vulnerability Detection (2021, background): “Another line of work, such as GAT [43] and GAAN [44], employs attention mechanisms to learn the weights of different neighboring nodes.”
How Attentive are Graph Attention Networks? (2021, methodology): “Transformers and GNNs Many works employed GNNs with Transformer-style attention (dotproduct) (Zhang et al., 2018; Thekumparampil et al., 2018; Gao and Ji, 2019; Lukovnikov and Fischer, 2021; Shi et al., 2020; Dwivedi and Bresson, 2020; Busbridge et al., 2019; Rong et al., 2020a; Veličković et…”
Revisiting Graph Neural Networks: All We Have is Low-Pass Filters (2019, methodology): “Started with the early success of ChebNet [5] and GCN [15] at vertex classification, many variants of GNN have been proposed to solve problems in social networks [11, 29], biology [24, 25], chemistry [7, 9], natural language processing [2, 30], computer vision [19], and weakly-supervised learning [8”
Graph Learning: A Survey (2021, background): “Likewise, Gated Attention Networks (GAANs) also introduced the multi-head attention mechanism for updating the hidden state of some vertices [144].”
BertGCN: Transductive Text Classification by Combining GNN and BERT (2021, background): “In the context of NLP, GNNs have achieved remarkable successes across a wide range of end tasks such as relation extraction (Zhang et al., 2018b), semantic role labeling (Marcheggiani and Titov, 2017), data-to-text generation (Marcheggiani and Perez-Beltrachini, 2018), machine translation (Bastings…”
GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks (2020, background): “Gated Attention Network (GaAN) [13] is a new network architecture that uses a convolutional sub-network to compute a soft gate at each attention head to control its importance, and the gated attention can modulate the amount of attended content via the introduced gates.”
MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding (2020, background): “GaAN [34] combines GRU with the gated multi-head attention mechanism for dealing with spatiotemporal graphs.”
A Review of Graph Neural Networks and Their Applications in Power Systems (2021, methodology): “Another popular variant is the gated attention network (GaAN) which employs the dot product attention and key-value attention mechanism [44].”