Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure

By Veselin Stoyanov, Alexander Ropson, Jason Eisner. 2011

132 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Structured Attention Networks (2017, methodology): “Past work has demonstrated the techniques necessary for this approach (see Stoyanov et al. (2011)), but to our knowledge it is very rarely employed.”
Gated Graph Sequence Neural Networks (2015, background): “An analogy can be drawn between our adaptation from GNNs to GG-NNs, to the work of Domke (2011) and Stoyanov et al. (2011) in the structured prediction setting.”
Representation Learning: A Review and New Perspectives (2012, background): “…of inference and learning (Bagnell and Bradley, 2009; Gregor and LeCun, 2010b; Grubb and Bagnell, 2010; Salakhutdinov and Larochelle, 2010; Stoyanov et al., 2011; Eisner, 2012), where a central idea is that instead of using a generic inference mechanism, one can use one that is learned…”
Conditional Random Fields as Recurrent Neural Networks (2015, background): “[54] showed how back-propagation through belief propagation can be used to optimize model parameters.”
Feedforward semantic segmentation with zoom-out features (2014, background): “that this can be done without giving up the feedforward nature of our approach; one possibility we are interested in exploring is to “unroll” approximate inference into additional layers in the feedforward network [23, 36].”
Deep Learning: Methods and Applications (2014, background): “A related study on using the discriminative criterion of empirical risk to train deep graphical models can be found in [352].”
OptNet: Differentiable Optimization as a Layer in Neural Networks (2017, background): “Among many other applications, these architectures are well-studied for generic classification and structured prediction tasks (Goodfellow et al., 2013; Stoyanov et al., 2011; Brakel et al., 2013; LeCun et al., 2006; Belanger & McCallum, 2016; Belanger et al., 2017; Amos et al., 2017); in vision for”
Deep Learning of Representations: Looking Forward (2013, methodology): “To go farther than Bengio and Thibodeau-Laufer (2013); Goodfellow et al. (2013a); Stoyanov et al. (2011) it would be good to go beyond the kind of factorized prediction common in variational and loopy belief propagation inference.”
Structured Prediction Energy Networks (2015, methodology): “Optimization over [0 , 1] L can be performed using projected gradient descent, or entropic mirror descent by normalizing over each coordinate (Beck & Teboulle, 2003).”
Iterative Neural Autoregressive Distribution Estimator NADE-k (2014, background): “This learning-to-cope was introduced recently in 2011 by Stoyanov et al. (2011) and Domke (2011).”
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures (2014, methodology): “In [11], gradient-based optimization of loopy belief propagation was applied to binary pair-wise Markov random ﬁelds.”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “A related work on using the discriminative criterion of empirical risk to train deep graphical models can be found in [81].”
Joint Training of Generic CNN-CRF Models with Stochastic Optimization (2015, background): “Besides the approaches [11] and [12], there are many other works that consider the idea of backpropagation with a so-called unrolled CRF-inference scheme, such as [14,15,16,17,18,19,20].”
Efficient Likelihood Learning of a Generic CNN-CRF Model for Semantic Segmentation (2015, background): “l class of pairwise potentials only. Besides the approaches [11] and [12], there are many other works that consider the idea of backpropagation with a so-called unrolled CRF-inference scheme, such as [14,15,16,17,18,19,20]. These inference steps mostly correspond to message passing operations of e.g”
Task-based End-to-end Model Learning (2017, unknown): “For example, both Stoyanov et al. (2011) and Hazan et al. (2010) proposed methods for optimizing loss criteria in structured prediction that are different from the inference procedure of the prediction algorithm; the work has also been recently extended to the setting of deep networks (Song et al.,…”
Differentiable Convex Optimization Layers (2019, background): “It is possible to differentiate through nonconvex problems, either analytically [37, 67, 5] or by unrolling SGD [33, 14, 61, 41, 70, 23, 38], Because convex programs can typically be solved efﬁciently and to high accuracy, it is preferable to use convex optimization layers over nonconvex…”