Inference in Probabilistic Graphical Models by Graph Neural Networks

By Kijung Yoon, Renjie Liao, Yuwen Xiong, Lisa Zhang, Ethan Fetaya, Raquel Urtasun, Richard S. Zemel, Xaq Pitkow. 2018

110 citations recorded by the source corpus.

A fundamental computation for statistical inference and accurate decision-making is to estimate the marginal probabilities or most probable states of task-relevant variables. Probabilistic graphical models can efficiently represent the structure of such complex data, but performing these inferences is generally difficult. Message-passing algorithms, such as belief propagation, are a natural way to disseminate evidence amongst correlated variables while exploiting the graph structure, but these algorithms can struggle when the conditional dependency graphs contain loops. Here we use Graph Neural Networks (GNNs) to learn a message-passing algorithm that solves these inference tasks. We first show that the architecture of GNNs is well-matched to inference tasks. We then demonstrate the efficacy of this inference approach by training GNNs on a collection of graphical models and showing that they substantially outperform belief propagation on loopy graphs. Our message-passing algorithms generalize out of the training set to larger graphs and graphs with different structure.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Relational inductive biases, deep learning, and graph networks (2018, background): “…Nowak et al., 2017; Dai et al., 2017), boolean satisﬁability (Selsam et al., 2018), program representation and veriﬁcation (Allamanis et al., 2018; Li et al., 2016), modeling cellular automata and Turing machines (Johnson, 2017), and performing inference in graphical models (Yoon et al., 2018).”
Graphite: Iterative Generative Modeling of Graphs (2018, background): “, 2016), formalizing what has thus far has been largely speculated empirically to the best of our knowledge (Yoon et al., 2018).”
LanczosNet: Multi-Scale Deep Graph Convolutional Networks (2019, background): “Moreover, many applications have recently emerged for GNNs, including community detection [41], situation recognition [42], RGBD semantic segmentation [43], few-shot learning [21], probabilistic inference [44], continuous control of reinforcement learning [45, 46] and so on.”
Mutual CRF-GNN for Few-shot Learning (2021, background): “We attribute the improvement difference to the inaccurate marginal distribution estimation by loopy belief propagation [37] when CRF has dense connections and a large number of nodes [57].”
Learning to Reason: Leveraging Neural Networks for Approximate DNF Counting (2019, background): “There has also been work towards outright learning of inference in probabilistic graphical models (Yoon et al. 2018).”
GMNN: Graph Markov Neural Networks (2019, result): “There are also some recent studies (Yoon et al., 2018) using graph neural networks for inference in probabilistic graphical models.”