GMNN: Graph Markov Neural Networks

By Meng Qu, Yoshua Bengio, Jian Tang. 2019

317 citations recorded by the source corpus.

This paper studies semi-supervised object classification in relational data, which is a fundamental problem in relational data modeling. The problem has been extensively studied in the literature of both statistical relational learning (e.g. relational Markov networks) and graph neural networks (e.g. graph convolutional networks). Statistical relational learning methods can effectively model the dependency of object labels through conditional random fields for collective classification, whereas graph neural networks learn effective object representations for classification through end-to-end training. In this paper, we propose the Graph Markov Neural Network (GMNN) that combines the advantages of both worlds. A GMNN models the joint distribution of object labels with a conditional random field, which can be effectively trained with the variational EM algorithm. In the E-step, one graph neural network learns effective object representations for approximating the posterior distributions of object labels. In the M-step, another graph neural network is used to model the local label dependency. Experiments on object classification, link classification, and unsupervised node representation learning show that GMNN achieves state-of-the-art results.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

A Gentle Introduction to Deep Learning for Graphs (2019, methodology): “ ✗ ✗ ✗ ECC [111] Continuous Topological ✗ ✗ GraphSAGE [54] ✗ ✗ ✗ ✓ CGMM [3] Discrete ✗ ✗ ✗ DiﬀPool [137] - Adaptive - - DGCNN [142] ✗ Topological ✗ ✗ GAT [120] ✗ ✗ ✓ ✗ R-GCN [105] Discrete ✗ ✗ ✗ GMNN [97] - - - - DGI [121] ✗ ✗ ✗ ✓ GIN [131] ✗ ✗ ✗ ✗ NMFPool [2] - Topological - - SAGPool [76] - Adapti”
Graph Representation Learning (2020, background): “…of message passing can be derived based on different approximate inference algorithms (e.g., Bethe approximations as discussed in Dai et al. [2016]), and there are also several works which explore how GNNs can be integrated more generally into PGM models [Qu et al., 2019, Zhang et al., 2020].”
Combining Label Propagation and Simple Models Out-performs Graph Neural Networks (2020, methodology): “Recent research connects GNNs to label propagation (Wang & Leskovec, 2020; Jia & Benson, 2020) as well as Markov Random ﬁelds (Qu et al., 2019; Gao et al., 2019), and some techniques use ad hoc incorporation of label information in the features (Shi et al., 2020).”
Unifying Graph Convolutional Neural Networks and Label Propagation (2020, background): “…the node side V , node features are propagated along edges and transformed through neural network layers, which is known as Graph Convolutional Neural Networks (GCN) (Kipf & Welling, 2017; Hamilton et al., 2017; Li et al., 2018; Xu et al., 2018; Liao et al., 2019; Xu et al., 2019; Qu et al., 2019).”
A Note on Over-Smoothing for Graph Neural Networks (2020, methodology): “…performance on semi-supervised classiﬁcation, several variants of GNNs have been developed for this task, including Graph-SAGE (Hamilton et al., 2017), GAT (Veli ˇ ckovi ´ c et al., 2017), SGC (Wu et al., 2019), CGCNN (Xie & Grossman, 2018) and GMNN (Qu et al., 2019) to name a few most recent ones.”
PairNorm: Tackling Oversmoothing in GNNs (2019, background): “…2017) on achieving state-of-the-art performance on semi-supervised classiﬁcation, several variants of GNNs have been developed for this task; including GraphSAGE (Hamilton et al., 2017), GAT (Velickovic et al., 2018), SGC (Wu et al., 2019), and GMNN (Qu et al., 2019) to name a few most recent ones.”
A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning (2021, methodology): “A GMNN is able to learn an effective node representation and model label dependency between different nodes, thereby completing the task of semi-supervised node classiﬁcation.”
Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs (2020, background): “[27] Meng Qu, Yoshua Bengio, and Jian Tang.”