Simple and Deep Graph Convolutional Networks

By Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, Yaliang Li. 2020

1,998 citations recorded by the source corpus.

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Recently, GCNs and subsequent variants have shown superior performance in various application areas on real-world datasets. Despite their success, most of the current GCN models are shallow, due to the {\em over-smoothing} problem. In this paper, we study the problem of designing and analyzing deep graph convolutional networks. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: {\em Initial residual} and {\em Identity mapping}. We provide theoretical and empirical evidence that the two techniques effectively relieves the problem of over-smoothing. Our experiments show that the deep GCNII model outperforms the state-of-the-art methods on various semi- and full-supervised tasks. Code is available at this https URL .

Subfields: Artificial Intelligence, Information Systems, Model Architectures, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Model Architectures

In Graph Learning

Citations in the corpus

Auto-GNN: Neural architecture search of graph neural networks (2019, background): “Recently, the deep GNNs have been widely studied to learn the high-order neighborhood structures of nodes (Chen et al., 2020; Zhou et al., 2020).”
Masked Label Prediction: Unified Massage Passing Model for Semi-Supervised Classification (2020, methodology): “37% absolute improvements compared to GCNII [Chen et al., 2020], whose parameters are four times larger than ours.”
UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation (2021, methodology): “According to Theorem 1 in [5], we can derive the infinite powers of message passing which take the following limit:”
Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods (2021, neutral): “The complexity of GCNII [15] is asymptotically the same as that of GCN, but in practice it requires more computations per layer due to residual connections and linear combinations, and it also often achieves best performance with a large number of layers L .”
Node Similarity Preserving Graph Convolutional Networks (2020, methodology): “We reuse the metrics reported in [5] for GCN, GAT and Geom-GCN.”
A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation (2022, methodology): “The idea is derived from recent researches on model sparsification [23] and simplification [14].”
Long Range Graph Benchmark (2022, methodology): “This points towards the proposed benchmarks being different from several classical benchmarks such as Cora or Citeseer, inter alia [11], where shallow GCNs [33] fared better than deeper GCNs.”
DyGRAIN: An Incremental Learning Framework for Dynamic Graphs (2022, methodology): “Furthermore, we compare our method using four GNN-based models, GCN [Kipf and Welling, 2017], GAT [Veliˇckovi´c et al. , 2018], APPNP [Klicpera et al. , 2019], and GCN2 [Ming Chen et al. , 2020] as a base learner model.”
Is Homophily a Necessity for Graph Neural Networks? (2021, methodology): “Following previous works (Li et al., 2018; Chen et al., 2020; Baranwal et al., 2021), we drop the non-linearity in the analysis.”
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks (2021, methodology): “, 2019); and a new model that even increases performance as more layers are stacked (Chen et al., 2020).”
Implicit Graph Neural Networks (2020, background): “Since then, a line of empirical (Li et al., 2018; Chen et al., 2020; Rong et al., 2020) and theoretical (Oono and Suzuki, 2020; Zhao and Akoglu, 2020) works follows on the over-smoothing phenomenon.”
Towards Deeper Graph Neural Networks with Differentiable Group Normalization (2020, methodology): “Recently, model GCNII∗ [39] has been proposed to prevent the over-smoothing by the residual connection and the identity mapping, which achieves the new state-of-the-art results with a 64-layer architecture.”
A Survey on Oversmoothing in Graph Neural Networks (2023, neutral): “To this end, we choose two representative methods of each of the different strategies to overcome over-smoothing, namely DropEdge and PairNorm as representatives from “normalization and regularization” strategies, GraphCON and G 2 from “change of GNN dynamics”, and Residual GCN (Res-GCN) and GCNII f”
Finding Global Homophily in Graph Neural Networks When Meeting Heterophily (2022, neutral): “We compare GloGNN and GloGNN++ with 11 other baselines, including (1) MLP; (2) general GNN methods: GCN (Kipf & Welling, 2016), GAT (Veličković et al., 2017), MixHop (Abu-El-Haija et al., 2019) and GCNII (Chen et al., 2020); (3) heterophilous-graph-oriented methods: H2GCN (Zhu et al., 2020b), WRGA”
Language is All a Graph Needs (2023, methodology): “…(He et al., 2023), we select those top-ranked GNNs from the two corresponding benchmarks with 60%/20%/20% train/val/test splits, including Snowball, RevGAT, etc. (Abu-El-Haija et al., 2019; Pei et al., 2020; Wu et al., 2019; He et al., 2021; Bo et al., 2021; Chen et al., 2020b; Luan et al., 2022).”
Graph Neural Networks with Heterophily (2020, neutral): “• GCN-Cheby (Defferrard, Bresson, and Vandergheynst 2016).”
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation (2021, methodology): “We construct a spectral domain graph convolutional network to encode the multi-modal contextual information inspired by (Li et al., 2019; Chen et al., 2020).”