Estimating the "Wrong" Graphical Model: Benefits in the Computation-Limited Setting

By Martin J. Wainwright. 2006

129 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Stable and Fair Classification (2019, background): “The generalization bound, i.e., the quality of the estimation | R ( A S ) − E ( A S ) | , depends on the number of samples N and algorithm A , and has been well studied in the literature [1, 10, 71, 48].”
Learning message-passing inference machines for structured prediction (2011, methodology): “Additionally, it has been observed that it is important to tie the graphical model to the specific approximate inference procedure used at test time to obtain better predictions [10, 22].”
Stacked Hierarchical Labeling (2010, methodology): “As we are forced to use approximate inference during learning, the learned model ( e.g. , parameters) is tightly tied to the chosen inference procedure in both theory [29] and in practice [17].”
Blending Learning and Inference in Conditional Random Fields (2016, background): “This validates the intuition behind the theoretical results of Wainwright et al. (2003); Wainwright (2006) which asserts that whenever learning and inference occur together one can use pseudo moment matching for learning the parameters.”
Stability and Generalization in Structured Prediction (2016, background): “Wainwright (2006) analyzed the stability of marginal probabilities in variational inference, identifying the relationship between stability and strong convexity (similar to our work in London et al., 2013, 2014).”
Parameter Learning and Convergent Inference for Dense Random Fields (2013, methodology): “While it can be shown that the approximate gradient obtained in this way corresponds to a welldefined surrogate objective (Wainwright, 2006), it may not optimize the performance of the learned model on the data (Kulesza & Pereira, 2007).”
Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure (2011, methodology): “Guided by the intuition that the errors of approximate methods in the estimation and prediction phases may cancel one another, Wainwright (2006) provides theoretical analysis to show that it is beneficial with respect to end-to-end performance to learn the “wrong” model by using inconsistent methods”
A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction (2010, methodology): “In particular, [24] proved that using a concave entropy approximation gives robust prediction.”
Structured Learning with Approximate Inference (2007, unknown): “If (xk, zk) is the instance on which the kth update occurs and zLP(k) = LPwk(xk), then by the update rule, ‖wk+1‖2 = ‖wk‖2 + 2wk · (Φ(xk, zk)− Φ(xk, zLP(k))) + ‖Φ(xk, zk)− Φ(xk, zLP(k))‖2 ≤ ‖wk‖2 +R2 .”
Recurrent Inference Machines for Solving Inverse Problems (2017, background): “First, it burdens practitioners with the choice of appropriate prior and inference models for the given data and task at hand, which typically requires a lot of expert knowledge from practitioners.”
Minimum-Risk Training of Approximate CRF-Based NLP Systems (2012, background): “Wainwright (2006) showed that if approximate inference will be used at test time, it may be beneﬁcial to use a learning procedure that does not converge to the true model but to one that performs well under the approximations .”
The Benefits of Learning with Strongly Convex Approximate Inference (2015, background): “Unfortunately, for general graph structures, M may require an exponential number of constraints, and Φ∗ may (2)See Wainwright & Jordan (2008) for a precise definition.”
Distributed Parameter Estimation via Pseudo-likelihood (2012, methodology): “Another line of work approximates MLE by estimating the partition function with variational algorithms (e.g., Wainwright, 2006; Sutton & McCallum, 2009).”
PAC-Bayesian Collective Stability (2014, background): “An example of this class is the “convexified” Bethe approximation (Wainwright, 2006).”
Learning Graphical Model Parameters with Approximate Marginal Inference (2013, methodology): “This “surrogate likelihood” [17] approximation appears to be the most widely used loss in imaging problems, with marginals approximated by either mean ﬁeld [18], [19], TRW [20] or LBP [21], [22], [23], [24], [25].”
Collective Stability in Structured Prediction: Generalization from One Example (2013, methodology): “This identity is commonly used to perform approximate marginal inference, by relaxing the marginal polytope and using a convex surrogate for −H (Wainwright, 2006).”
Structure Regularization for Structured Prediction (2014, background): “Let Y n ⊂ R n be structured output space, so that the structured output yyy are sampled from Y n = n , Let Z ( , Y n ) be a uniﬁed denotation of structured input and output space. is from X Let zzz = ( xxx,yyy ) which sampled Z , be a uniﬁed denotation of a ( xxx,yyy ) pair in the training data.”
Graphical Models, Exponential Families, and Variational Inference (2008, methodology): “11) is strongly convex [236] for any pairwise Markov random field, so that as a corollary, the treereweighted sum-product algorithm is guaranteed to be globally Lipschitz stable.”