Deeply Learning the Messages in Message Passing Inference

By Guosheng Lin, Chunhua Shen, Ian David Reid, Anton van den Hengel. 2015

64 citations recorded by the source corpus.

Deep structured output learning shows great promise in tasks like semantic image segmentation. We proffer a new, efficient deep structured model learning scheme, in which we show how deep Convolutional Neural Networks (CNNs) can be used to directly estimate the messages in message passing inference for structured prediction with Conditional Random Fields (CRFs). With such CNN message estimators, we obviate the need to learn or evaluate potential functions for message calculation. This confers significant efficiency for learning, since otherwise when performing structured learning for a CRF with CNN potentials it is necessary to undertake expensive inference for every stochastic gradient iteration. The network output dimension of message estimators is the same as the number of classes, rather than exponentially growing in the order of the potentials. Hence it is more scalable for cases that involve a large number of classes. We apply our method to semantic image segmentation and achieve impressive performance, which demonstrates the effectiveness and usefulness of our CNN message learning method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “In these existing messages learning work (Hershey et al., 2014; Zheng et al., 2015; Lin et al., 2015), the learning task is still estimating the messages represented graphical models with designed function forms, e.”
Learning Affinity via Spatial Propagation Networks (2017, methodology): “Some methods use CFR as a post-processing module [5], while others integrate it as a jointly-trained part [36, 17, 24, 1].”
Recurrent Relational Networks (2017, background): “[Khan et al., 2014] 17 92.5% Loopy BP, random [Bauke, 2008] 17 61.7% Loopy BP, parallel [Bauke, 2008] 17 53.2% Deeply Learned Messages* [Lin et al., 2015] 17 0% Relational Network, node* [Santoro et al., 2017] 17 0% Relational Network, graph* [Santoro et al., 2017] 17 0% Deep Convolutional Network…”
CRF-CNN: Modeling Structured Information in Human Pose Estimation (2016, unknown): “Our implementation is as follows: En(z,h, I,Θ) = ∑ (i,j)∈Eh i<j ψh(hi,hj) + ∑ (i,j)∈Ez i<j ψz(zi, zj) + N∑ i=1 ψzh(zi,hi) + K∑ k=1 φh(hk, I), (9)”
Inference in Probabilistic Graphical Models by Graph Neural Networks (2018, background): “[10] proposes to train a neural network that learns to map message inputs to message outputs for each message operation needed for Expectation Propagation inference, and [11] suggests learning CNNs for estimating factor-to-variable messages in a message-passing procedure.”
Fast Semantic Image Segmentation with High Order Context and Guided Filtering (2016, background): “Another probability of encoding context information is learning the messages based on feature context [17,9].”
Deep Structured Prediction with Nonlinear Output Transformations (2018, unknown): “[29] proposed a unified formulation (see Fig.”
Gaussian Conditional Random Field Network for Semantic Segmentation (2016, background): “While all these approaches learn CNN-based potentials and use message passing algorithms to perform CRF inference, [27] has recently proposed to use CNNs to directly learn the messages in message passing inference.”
Survey on semantic segmentation using deep learning techniques (2019, methodology): “Instead of learning the potentials, [168] proposes a method that learns CNN message estimators for the message passing inference for structured Conditional Random Field (CRFs) predictions.”
Learning High-level Prior with Convolutional Neural Networks for Semantic Segmentation (2015, methodology): “To mitigate the predication inconsistency between pixels with similar appearance, recent works introduce Conditional Random Field (CRF) into the Neural Network framework [39, 30, 22, 28, 21].”
Learning to Filter with Predictive State Inference Machines (2015, background): “Instead of simply predicting next observation, we focus on predictive state—a distribution of future observations, as an alternative representation of the beliefs of latent states.”
Adversarial Deep Structural Networks for Mammographic Mass Segmentation (2016, background): “further explores CNN as the pairwise potential function in CRF capturing semantic correlations of neighboring patches [17, 16].”
Recurrent Relational Networks for Complex Relational Reasoning (2018, background): “Several works have proposed replacing parts of belief propagation with learned modules (Heess et al., 2013; Lin et al., 2015).”
Reasoning Visual Dialogs With Structural and Partial Observations (2019, methodology): “Some of these methods [30, 55, 4, 17, 26, 48] implement every graph node as a small neural network and formulate the interactions between nodes as a message propagation process, which is designed to be end-to-end trainable.”