Residual Gated Graph ConvNets

By Xavier Bresson, Thomas Laurent. arXiv.org, 2017

626 citations recorded by the source corpus.

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/a5fe578a6b9f51ce19263676e6395421fedc6d2d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Materials Chemistry

Citations in the corpus

Combinatorial optimization and reasoning with graph neural networks (2021, background): “[61] proposed the use of residual gated graph convolutional networks [13] in a supervised manner to solve the TSP.”
A Generalization of Transformer Networks to Graphs (2020, methodology): “…it is ideal and practical to have a Graph Transformer where a node attends to local node neighbors, same as in GNNs (Defferrard, Bresson, and Van-dergheynst 2016; Kipf and Welling 2017; Monti et al. 2017; Gilmer et al. 2017; Veliˇckovi´c et al. 2018; Bresson and Laurent 2017; Xu et al. 2019).”
Graph Neural Networks with Learnable Structural and Positional Representations (2021, methodology): “In our demonstrations, we formulate LSPE instances of both sparse GNNs, such as GatedGCNs (Bresson & Laurent, 2017) and PNA (Corso et al., 2020) and fully-connected Transformers-based GNNs (Kreuzer et al., 2021; Mialon et al., 2021).”
Benchmarking Graph Neural Networks (2023, methodology): “GatedGCN Layer Gated Graph ConvNet (GatedGCN) (Bresson & Laurent, 2017) GatedGCN considers residual connections, batch normalization and edge gates to design another anisotropic variant of GCN.”
Incremental Training of Graph Neural Networks on Temporal Graphs under Distribution Shift (2020, background): “Instances of anisotropic GNNs include graph attention networks (GAT) [28], GatedGCN [2] and MoNet [17].”
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks (2019, background): “Motivated by the natural graph-based representation for a series of skeletal forms and inspired by recent advances in graph convolution networks (GCNs) [9, 20, 41, 50], in this work, we propose to utilize GCNs to exploit spatial and temporal relationships for 3D pose estimation.”
Rethinking Graph Neural Architecture Search from Message-passing (2021, background): “Theoretically, recent popular GNNs (GCN [12], GIN [32], GraphSage [9], GAT [29], GatedGCN [4], etc.”
NeuGraph: Parallel Deep Neural Network Computation on Large Graphs (2019, methodology): “NeuGraph can support many different types of graph-based neural networks [7,8,13,18,19,23,25,29,41].”
Long Range Graph Benchmark (2022, neutral): “For the baselines, we select GCN [33], GCNII [11], GINE [62, 28] and GatedGCN [8] models from the local MP-GNN class, and fully connected Transformer [58] with Laplacian PE (LapPE) [14, 13] and SAN [34] models from the Transformer class.”
An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem (2019, methodology): “For this work, we leverage the graph ConvNet architecture introduced in Bresson and Laurent [2017] by deﬁning node features x +1 i and edge features e +1 ij as follows: where W ∈ R h × h , σ is the sigmoid function, ε is a small value, ReLU is the rectiﬁed linear unit, and BN stands for batch…”
Rethinking Graph Transformers with Spectral Attention (2021, methodology): “[15, 21], against GCN [25], GraphSage [18], GIN [39], GAT [37], GatedGCN [5], PNA [11], and DGN [4].”
Recipe for a General, Powerful, Scalable Graph Transformer (2022, methodology): “We also ﬁnd that although a vanilla PNA [13] generally outperforms GINE [26] and GatedGCN [7], adding the PE and SE results in major performance boost especially for the GatedGCN.”
Weisfeiler and Lehman Go Cellular: CW Networks (2021, neutral): “[11] Xavier Bresson and Thomas Laurent.”
Do Transformers Really Perform Bad for Graph Representation? (2021, neutral): “…[54] 509,549 0.526 ± 0.051 GraphSage [18] 505,341 0.398 ± 0.002 GAT [50] 531,345 0.384 ± 0.007 GCN [26] 505,079 0.367 ± 0.011 GatedGCN-PE [4] 505,011 0.214 ± 0.006 MPNN (sum) [15] 480,805 0.145 ± 0.007 PNA [10] 387,155 0.142 ± 0.010 GT [13] 588,929 0.226 ± 0.014 SAN [28] 508, 577 0.139 ±…”
DeepST: identifying spatial domains in spatial transcriptomics by deep learning (2022, background): “GCNConv and ResGatedGraphConv (29) obtain higher ARI values and better model robustness, relatively.”
Principal Neighbourhood Aggregation for Graph Nets (2020, neutral): “(GCN[15], GIN[6], DiffPool[32], GAT[16], MoNet[33] and GatedGCN[34]).”
Identity-aware Graph Neural Networks (2021, background): “We emphasize that ID-GNNs are fundamentally different from GNNs based on anisotropic message passing, where different attention weights are applied to different incoming edges (Bresson and Laurent 2017; Hamilton, Ying, and Leskovec 2017; Monti et al. 2017; Velickovic et al. 2018).”