Computing upper and lower bounds on likelihoods in intractable networks

By Tommi S. Jaakkola, Michael I. Jordan. 1996

We present deterministic techniques for computing upper and lower bounds on marginal probabilities in sigmoid and noisy-OR networks. These techniques become useful when the size of the network (or clique size) precludes exact computations. We illustrate the tightness of the bounds by numerical experiments.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Uncertainty-guided Continual Learning with Bayesian Neural Networks (2019, background): “Alternatively, variational inference provides a faster solution to the same problem in which the posterior is approximated using optimization rather than being sampled from a chain [30, 11, 13, 12].”
Mean Field Theory for Sigmoid Belief Networks (1996, methodology): “Recently, Jaakkola and Jordan (1996b) have developed a mean eld approximation for noisy-OR belief networks.”
Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching (2003, background): “This quantity plays a fundamental role in various contexts, including approximate inference [4], maximum likelihood parameter estimation [5], and large deviations analysis [3].”
Filtering Variational Objectives (2017, background): “an or their relationship non-linear. 2.2 The Evidence Lower Bound (ELBO) One strategy to circumvent the difﬁculty of marginalization is to optimize a lower bound (ELBO) on the marginal log-likelihood [27, 28, 29, 11, 12]. The ELBO is deﬁned by a second distribution q, the variational posterior, whic”
A Tighter Bound for Graphical Models (2001, methodology): “An improvement of this approximation was found by Thouless, Anderson, and Palmer (1977), which was applied to Boltzmann machines in Kappen and Rodrı́guez (1999). Unfortunately, this so-called TAP approximation is not a bound. We apply our method to the mean-field approximation, which results in a th”
An Introduction to Variational Methods for Graphical Models (1999, methodology): “Saul, Jaakkola, and Jordan (1996) and Saul and Jordan (this volume) also show how to update the variational parameters i.”
A Variational Approach to Bayesian Logistic Regression Models and their Extensions (1997, methodology): “If the variational distribution is chosen to be completely factorized the inequality corresponds to a mean eld approximation (see e.g. Saul et al. 1996, or Jaakkola & Jordan 1996).”
A new class of upper bounds on the log partition function (2002, methodology): “illsky@mit.edu Abstract Bounds on the log partition function are im portant in a variety of contexts, including ap proximate inference, model fitting, decision theory, and large deviations analysis [11, 5, 4]. We introduce a new class of upper bounds on the log partition function, based on con vex c”
A Tutorial on Learning with Bayesian Networks (1999, methodology): “For a discussion of selective model averaging, see Madigan and Raftery (1994).”
Bayesian Networks for Data Mining (2004, methodology): “…are developing techniques that are custom tailored to particular network topologies (Heckerman 1989; Suermondt and Cooper, 1991; Saul et al., 1996; Jaakkola and Jordan, 1996) or to particular inference queries (Ramamurthi and Agogino, 1988; Shachter et al., 1990; Jensen and Andersen, 1990;…”
An Auxiliary Variational Method (2004, background): “Their other advantage for graphical models is availability of rigorous bounds on the normalizing constant (Jaakkola and Jordan, 1996).”
Learning Bayesian networks: approaches and issues (2011, background): “Jaakkola and Jordan (1996) look at computing upper and lower bounds on likelihoods in sigmoid and noisy-OR networks.”