Structured Learning with Approximate Inference

By Alex Kulesza, Fernando C Pereira. 2007

155 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, background): “independent3 of each other given x), or has only a few modes between which it is easy to mix (MCMC). However, approximate inference can be potentially hurtful, not just at test time but for training (Kulesza and Pereira, 2008), because it is often in the inner loop of the learning procedure. We want”
Determinantal Point Processes for Machine Learning (2012, methodology): “What makes DPPs particularly unique is the marriage of these computational advantages with the ability to express global, negative interactions between modeling variables; this repulsive domain is notoriously intractable using traditional approaches like graphical models [17, 70, 82, 109, 146, 156, ”
Deep Learning of Representations: Looking Forward (2013, background): “This might be important because the compounding of approximate inference with function approximation could be very hurtful (Kulesza and Pereira, 2008).”
Structured Prediction Energy Networks (2015, neutral): “Using the LP relaxation, instead of exact MAP inference, in the inner loop of CRF SSVM learning is fairly benign, since it is guaranteed to over-generate margin violations in (8) (Kulesza & Pereira, 2007; Finley & Joachims, 2008).”
Learning message-passing inference machines for structured prediction (2011, background): “Unfortunately, learning these models with approximate inference is not well understood [9, 6].”
Pose Machines: Articulated Pose Estimation via Inference Machines (2014, background): “, loopy belief propagation), which makes parameter learning difficult [7].”
Learning Mixtures of Submodular Shells with Application to Document Summarization (2012, methodology): “However, as mentioned above, when an approximate inference algorithm is used in a learning algorithm, a good approximation of the score might not be sufficient, and it is possible that the learning can fail even with rigorous approximate guarantees [24].”
Efficient Structured Prediction with Latent Variables for General Graphical Models (2012, methodology): “…algorithms on structured SVMs (Taskar et al., 2004; 2005; Tsochantaridis et al., 2004) without latent variables has been investigated by (Finley & Joachims, 2008; Kulesza & Pereira, 2008), where they reported a “generally poor performance” when combining belief propagation and structured SVMs.”
Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure (2011, background): “Inference also plays a key role in parameter estimation, and Kulesza and Pereira (2008) show that approximate inference here can lead to pathological learning.”
Pylon Model for Semantic Segmentation (2011, background): “not easily achievable with the approximate and slow inference in previous pool-based methods [17].”
Tractable Semi-supervised Learning of Complex Structured Prediction Models (2013, methodology): “Composite likelihood methods [20], Piecewise training [32], message-passing algorithm [10], and large-margin methods with approximate inference [30,6,18] have been proposed for approximate learning.”
AD3: alternating directions dual decomposition for MAP inference in graphical models (2015, background): “This is particularly useful if inference is embedded in learning, where it is more important to obtain a fractional solution of the relaxed primal than an approximate integer one (Kulesza and Pereira, 2007; Martins et al., 2009).”
Parameter Learning and Convergent Inference for Dense Random Fields (2013, methodology): “While it can be shown that the approximate gradient obtained in this way corresponds to a welldefined surrogate objective (Wainwright, 2006), it may not optimize the performance of the learned model on the data (Kulesza & Pereira, 2007).”
Discriminative Learning of Sum-Product Networks (2012, background): “Research suggests that approximate inference can make it harder to learn rich structured models [21].”
Minimum-Risk Training of Approximate CRF-Based NLP Systems (2012, background): “Alas, standard learning procedures assume exact inference: they do not compensate for approximations that will be used at test time, and can go surprisingly awry if approximate inference is used at training time (Kulesza and Pereira, 2008).”
Stacked Hierarchical Labeling (2010, background): “For example, Kulesza and Pereira [16] demonstrate a case where learning with bounded approximate inference can prevent the learning procedure from ever reaching a feasible zero empirical risk solution.”
Structured Learning and Prediction in Computer Vision (2011, background): “In fact, recent results suggest that the use of suboptimal predictions during training can lead to prediction functions with bad performance [39, 86, 99], whereas alternative approaches based on relaxing the prediction problem work well in training.”
Learning to search: Functional gradient techniques for imitation learning (2009, methodology): “Perhaps the best known system built using this paradigm was ALVINN ( Pomerleau 1989 ).”