Constructing free-energy approximations and generalized belief propagation algorithms

By Jonathan S. Yedidia, William T. Freeman, Yair Weiss. 2005

1,756 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Molecular Biology, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Molecular Biology

In Probabilistic & Bayesian Methods

Citations in the corpus

Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “As we can see from the optimization problems (16) and (17), the objective functions are directly related to the corresponding discriminative tasks, and so as to W and u. Conceptually, the procedure starts with representing each datum by a graphical model constructed corresponding to its individual…”
Learning Deep Structured Models (2014, methodology): “In the future we plan to learn deeper models for object recognition in applications such as holistic semantic scene understanding.”
A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, background): “This derivation forms an optimisation problem with a resulting cost function widely known as variational free energy [50, 63, 12] which is built upon two terms: the former, KL [qθ(w|D)‖p(w)], is dependent on the definition of the prior p(w), thus called complexity cost, whereas the latter, Eq(w|θ)[l”
Kernel Belief Propagation (2011, background): “Following e.g. Yedidia et al. (2001, 2005) and Koller & Friedman (2009, Ch. 11), logZ may be approximated as a minimum of the Bethe free energy with respect to a new set of parameters {bst, bs}, F ({bst, bs}) = ∑ (s,t)∈E ∫ X ∫ X bst(Xs, Xt) [log bst(Xs, Xt)− log Ψst(Xs, Xt)Ψs(Xs)Ψt(Xt)] dXsdXt − ∑…”
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures (2014, methodology): “When it converges, the ﬁxed points correspond to stationary points of the Bethe free energy [17].”
Dependency Parsing by Belief Propagation (2008, methodology): “Our method and its numerical details emerge naturally as an instance of the well-studied loopy BP algorithm, suggesting several future improvements to accuracy (Yedidia et al., 2004; Braunstein et al., 2005) and efﬁciency (Sutton and McCallum, 2007).”
Learning in Markov Random Fields with Contrastive Free Energies (2005, methodology): “In the following we will be working with clusters consisting of edges and nodes only, called the “Bethe approximation”, but we like to emphasize that the formalism is easily adapted to general Kikuchi approximations, or in fact region graph approximations [19].”
Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference (2018, background): “This derivation forms an optimization problem with a resulting cost function widely known as variational free energy [27, 28, 29] which is built upon two terms: the former, KL [qθ(w|D)‖p(w)], is dependent on the definition of the prior p(w), thus called complexity cost, whereas the latter, Eq(w|θ)[l”
A new class of upper bounds on the log partition function (2002, methodology): “Finally, we demonstrate using a simple example the link between convex combinations of hypertrees and “convexified” forms of Kikuchi free energies [46], [47].”
A Linear Programming Approach to Max-Sum Problem: A Review (2007, neutral): “See, e.g., the introduction to [Yed04].”
Message passing algorithms for compressed sensing: I. motivation and construction (2009, background): “While the connection between belief propagation and Bethe-Peierls approximation stimulated a considerable amount of research [12], the algorithmic uses of TAP equations have received only sparse attention.”
Sparse-graph codes for quantum error correction (2003, background): “s B, N, and M, our decoder ignores these four cycles. It seems highly likely that a decoding algorithm that took these four-cycles into account would perform signiﬁcantly better (Yedidia et al. 2000; Yedidia et al. 2002). Our attempts to make such an improved algorithm have so far yielded only algor”
The Noise-Sensitivity Phase Transition in Compressed Sensing (2010, background): “em in spin glass theory, although important partial results exist [Tal03]. While the connection between belief propagation and Bethe-Peierls approximation stimulated a considerable amount of research [YFW05], the algorithmic uses of TAP equations have received only sparse attention. Remarkable 3When”
Collective Classification in Network Data (2008, methodology): “Mean-field relaxation labeling (Weiss 2001; Yedidia, Freeman, and Weiss 2005), discussed in this article, is a simple instance of this general class of algorithms.”