Learning message-passing inference machines for structured prediction

By Stéphane Ross, Daniel Munoz, Hebert J Andrew Bagnell. 2011

122 citations recorded by the source corpus.

Nearly every structured prediction problem in computer vision requires approximate inference due to large and complex dependencies among output labels. While graphical models provide a clean separation between modeling and inference, learning these models with approximate inference is not well understood. Furthermore, even if a good model is learned, predictions are often inaccurate due to approximations. In this work, instead of performing inference over a graphical model, we instead consider the inference procedure as a composition of predictors. Specifically, we focus on message-passing algorithms, such as Belief Propagation, and show how they can be viewed as procedures that sequentially predict label distributions at each node over a graph. Given labeled graphs, we can then train the sequence of predictors to output the correct labeling s. The result no longer corresponds to a graphical model but simply defines an inference procedure, with strong theoretical properties, that can be used to classify new graphs. We demonstrate the scalability and efficacy of our approach on 3D point cloud classification and 3D surface estimation from single images.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Natural Language Processing

In Model Architectures

Citations in the corpus

Convolutional Pose Machines (2016, methodology): “The overall architecture can be viewed as an unrolled mean-field message passing inference algorithm [31] that is learned end-to-end using backpropagation.”
Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “2 Comparison with Learning Message Estimator By recognizing inference as computational expressions, inference machines (Ross et al., 2011) incorporate learning into the messages passing inference for CRFs.”
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation (2014, methodology): “[26] use a message-passing inspired procedure for structured prediction on computer vision tasks, such as 3D point cloud classification and 3D surface estimation from single images.”
Reinforcement learning in robotics: A survey (2013, methodology): “Structured prediction, a key technique used within computer vision and robotics, where many predictions are made in concert by leveraging inter-relations between them, may be seen as a simplified variant of imitation learning (Daumé et al., 2009; Ross et al., 2011a).”
Higher Order Conditional Random Fields in Deep Neural Networks (2015, background): “Some early works that advocated gradient backpropagation through graphical model inference for parameter optimisation include [9, 19, 25] and [41].”
Iterative Instance Segmentation (2015, methodology): “Auto-context [32] and inference machines [26] take advantage of the observation that performing accurate inference does not necessarily require modelling the posterior distribution explicitly.”
Reinforcement and Imitation Learning via Interactive No-Regret Learning (2014, background): “[25] The resulting algorithms make suggestions for batch approaches as well: they suggest, for instance, that approximate policy iteration procedures (as well as imitation learning ones) are likely to be more stable and effective if they train not only on the cost-to-go of the most recent policy…”
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures (2014, background): “In a recent scene labeling paper [15], learning of belief propagation and mean ﬁeld message passing algorithms was considered for conditional random ﬁelds (CRFs), which are MRFs where all units are either inputs or labels.”
Recurrent Human Pose Estimation (2016, methodology): “In other related work, the iterated implicit configuration module of our model bears similarities to autocontext [40] and the message-passing inference machines of [32].”
Recurrent Relational Networks (2017, methodology): “An important insight from the work of Santoro et al. [2017] is to decompose a function for relational reasoning into two components or “modules”: a perceptual front-end, which is tasked to recognize objects in the raw input and represent them as vectors, and a relational reasoning module, which uses”
Pose Machines: Articulated Pose Estimation via Inference Machines (2014, methodology): “The use of outputs from all neighboring variables, resembles the message passing mechanics in variational mean field inference [9].”
Variational inference of latent state sequences using Recurrent Networks (2014, methodology): “To circumvent this, learning a model for fast approximate inference is an old idea [14] which has been applied in miscellaneous settings recently [24, 12, 23, 17].”
Deep Spatio-Temporal Architectures and Learning for Protein Structure Prediction (2012, background): “The idea of layering learning modules, such that the outputs of previous layers are fed in input to the next layers, is not completely new and it has been applied in different contexts, particularly to computer vision detection problems [4, 10, 12, 22].”
Proximal Deep Structured Models (2016, result): “This observation is consistent with the findings of shrinkage field [33] and inference machines [29].”
Contextual Priming and Feedback for Faster R-CNN (2016, background): “There have been attempts earlier at exploiting feedback mechanisms; some well known examples are auto-context [45] and inference machines [46].”
Recurrent Relational Networks for Complex Relational Reasoning (2018, background): “Ross et al. (2011) proposes Inference Machines which ditch the belief propagation algorithm altogether and instead train a series of regressors to output the correct marginals by passing messages on a graph.”
Learning to Pass Expectation Propagation Messages (2013, background): “…the most superficially similar line of work to the approach we present here is that of inference machines and truncated belief propagation [2, 3, 10, 9], where inference is done via an algorithm that is structurally similar to belief propagation, but where some parameters of the updates are…”
Higher Order Potentials in End-to-End Trainable Conditional Random Fields (2015, background): “Some early works that advocated gradient backpropagation through graphical model inference for parameter optimization include [35, 8, 22] and [17].”
Efficient Likelihood Learning of a Generic CNN-CRF Model for Semantic Segmentation (2015, background): “amp; Urtasun, 2015), there are many other works that consider the idea of backpropagation with a so-called unrolled CRF-inference scheme, such as (Domke, 2013; Kiefel & Gehler, 2014; Barbu, 2009; Ross et al., 2011; Stoyanov et al., 2011; Tompson et al., 2014; Liu et al., 2015). These inference s”