Exploiting Tractable Substructures in Intractable Networks

By Lawrence K. Saul, Michael I. Jordan. 1995

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Stochastic variational inference (2012, background): “In Michael Jordan’s lab, the seminal papers of Saul et al. (1996); Saul and Jordan (1996) and Jaakkola (1997) grew out of reading the statistical physics literature (Peterson and Anderson, 1987; Parisi, 1988).”
Variational Inference: A Review for Statisticians (2016, background): “As we mentioned previously, structured variational inference has its roots in the early days of the method (Saul and Jordan 1996; Barber and Wiegerinck 1999).”
The Variational Gaussian Process (2015, background): “Many expressive variational approximations can be viewed under this construct (Saul & Jordan, 1996; Jaakkola & Jordan, 1998; Rezende & Mohamed, 2015; Tran et al., 2015).”
Hierarchical Variational Models (2015, methodology): “Saul and Jordan (1996); Ghahramani (1997) develop structured variational approximations: they factorize the variational family across subsets of variables, maintaining certain dependencies in the model.”
Deep Learning of Representations: Looking Forward (2013, background): “To make the 16 this can be relaxed by considering tree-structured conditional dependencies (Saul and Jordan, 1996) and mixtures thereof mental picture simpler, imagine that there are only two kinds of probabilities: tiny and high.”
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression (2012, methodology): “Algorithm 1 Stochastic Optimization for Fixed-Form Variational Bayes Require: An unnormalized posterior distribution p ( x, y ) Require: A type of approximating posterior q η ( x ) Require: The total number of iterations N Initialize ˜ η 1 to a ﬁrst guess, for example by matching the prior p ( x ) o”
Tractable Variational Structures for Approximating Graphical Models (1998, methodology): “The process of using a Q defined on the whole graph but for which only a subset of the connections are adaptive is termed substructure variation [10].”
Quick Training of Probabilistic Neural Nets by Importance Sampling (2003, background): “neural networks, distributed representation graphical models (Saul and Jordan, 1996), or Maximum Entropy models (Berger, Della Pietra and Della Pietra, 1996)) can be exponentially more efficient than “localist” models, allowing to consider many higher-order dependencies.”
Copula variational inference (2015, background): “Less common, structured mean-field methods slightly relax this assumption by preserving some of the original structure among the latent variables [19].”
Bayesian parameter estimation via variational methods (2000, background): “More generally, one can consider structured variational distributions involving partial factorizations that correspond to tractable substructures in the graphical 32 Jaakkola and Jordan model (cf. Saul and Jordan 1996 ).”
Bayesian reasoning and machine learning (2012, unknown): “5 Structured variational approximation One can extend the factorised KL variational approximation by using non-factorised q(x)[239, 24].”
MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES (1996, unknown): “Future work will concentrate on a more e cient mean eld approximation in which the forward-backward algorithm is used to compute the E-step exactly within each Markov chain, and mean eld theory is used to handle interactions between chains (Saul and Jordan, 1996).”
Factorial Hidden Markov Models (1995, methodology): “Future work will concentrate on a more e cientmean eld approximation in which the forward-backward algorithm is used to com-pute the E-step exactly within each Markov chain, and mean eld theory is used tohandle interactions between chains (Saul and Jordan, 1996).”