Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

By Sanjiv Kumar, Jonas August, Martial Hebert. 2005

63 citations recorded by the source corpus.

Estimation of parameters of random field models from labeled training data is crucial for their good performance in many image analysis applications. In this paper, we present an approach for approximate maximum likelihood parameter learning in discriminative field models, which is based on approximating true expectations with simple piecewise constant functions constructed using inference techniques. Gradient ascent with these updates exhibits compelling limit cycle behavior which is tied closely to the number of errors made during inference. The performance of various approximations was evaluated with different inference techniques showing that the learned parameters lead to good classification performance so long as the method used for approximating the gradient is consistent with the inference mechanism. The proposed approach is general enough to be used for the training of, e.g., smoothing parameters of conventional Markov Random Fields (MRFs).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Structured Convolutional Kernel Networks for Airline Crew Scheduling (2021, methodology): “We ﬁrst run an approximate MAP inference algorithm and then imagine that the marginals put a unit probability at the approximate MAP solution and zero elsewhere (Kumar et al., 2005; Zhong & Wang, 2007; Scharstein & Pal, 2007).”
Learning message-passing inference machines for structured prediction (2011, methodology): “Additionally, it has been observed that it is important to tie the graphical model to the specific approximate inference procedure used at test time to obtain better predictions [10, 22].”
Learning CRFs Using Graph Cuts (2008, methodology): “Currently, the most widely used learning algorithms include cross-validation and simple partition function approximations [7].”
Contextual classification with functional Max-Margin Markov Networks (2009, methodology): “[12] to use the same approximate inferences during both training and testing.”
Learning optimized MAP estimates in continuously-valued MRF models (2009, unknown): “in [7].”
Probabilistic Joint Image Segmentation and Labeling (2011, methodology): “[22] found this approximation to perform best for learning c onditional random fields for pixel labeling.”
A Hybrid Approach to Detect and Localize Texts in Natural Scene Images (2011, methodology): “— Saddle Point Approximation (SPA) SPA is a measurement to approximate the probability distribution with the Maximum a posteriori (MAP) estimation which has been used to overcome the computational intractable problem [15].”
Stacked Hierarchical Labeling (2010, methodology): “As we are forced to use approximate inference during learning, the learned model ( e.g. , parameters) is tightly tied to the chosen inference procedure in both theory [29] and in practice [17].”
Discriminative Random Fields (2006, background): “We are currently exploring other possibilities of parameter learning, as discussed in our recent work (Kumar et al., 2005).”
The potential of automatic methods of classification to identify leaf diseases from multispectral images (2011, methodology): “Korc ˇ and Fo ¨rstner (2008) showed that parameter learning based on a pseudolikelihood approximate scheme in combination with the maximum a posteriori inference gives results comparable to the state of the art approaches in Kumar et al. (2005) .”
Recurrent Inference Machines for Solving Inverse Problems (2017, methodology): “In fact, it is well known that the optimization procedure used for training should match the one used during testing because the model has adapted itself to perform well under that optimization procedure [8, 9].”
Parameter Learning and Convergent Inference for Dense Random Fields (2013, methodology): “One way to deal with this is to approximate the gradient by substituting approximate inference in place of exact inference (Wainwright et al., 2003; Sutton & McCallum, 2005; Kumar et al., 2005; Vishwanathan et al., 2006; Levin & Weiss, 2009; Pal et al., 2012).”
Learning Graphical Model Parameters with Approximate Marginal Inference (2013, methodology): “This “surrogate likelihood” [17] approximation appears to be the most widely used loss in imaging problems, with marginals approximated by either mean ﬁeld [18], [19], TRW [20] or LBP [21], [22], [23], [24], [25].”