Graphical Models, Exponential Families, and Variational Inference

By Martin J. Wainwright, Michael I. Jordan. 2008

4,834 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Probabilistic & Bayesian Methods

Citations in the corpus

Discriminative Embeddings of Latent Variable Models for Structured Data (2016, positive): “Several theoretical studies have also provided insight into the approximations made by loopy BP, partially justifying its application to graphs with cycles (Wainwright & Jordan, 2008; Yedidia et al., 2001a).”
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “Our theory holds for a rich family of conditional densities p(zt i | z<t, a<t) called the exponential family (Wainwright and Jordan, 2008), which has the following form: p(z i | z, a) = hi(z i) exp{Ti(z i)λi(Gi z, Gi a)− ψi(z, a)} .”
An Analysis of the Adaptation Speed of Causal Models (2020, methodology): “We study two of the simplest sub-families of the exponential family (Wainwright and Jordan, 2008): categorical and linear normal variables.”
On the Equivalence between Herding and Conditional Gradient Algorithms (2012, methodology): “We denote by M ⊂ F the marginal polytope (Wainwright & Jordan, 2008; Chen et al., 2010), i.e., the convex-hull of all vectors Φ( x ) for x ∈ X .”
Meta-Learning Probabilistic Inference for Prediction (2018, background): “For a single task t , an evidence lower bound (ELBO; (Wainwright and Jordan, 2008)) may be expressed as:   We can then derive a stochastic estimator to optimize Eq.”
Learning Dependency-Based Compositional Semantics (2011, background): “A standard result is that the derivative of the log-partition function is the expected feature vector (Wainwright and Jordan, 2008).”
Black Box Variational Inference (2013, methodology): “Variational inference transforms the problem of approximating a conditional distribution into an optimization problem (Jordan et al., 1999; Bishop, 2006; Wainwright and Jordan, 2008).”
Unifying Count-Based Exploration and Intrinsic Motivation (2016, background): “We next show that directed graphical models (Wainwright and Jordan, 2008) satisfy Assumption 1.”
Stochastic variational inference (2012, methodology): “Traditionally, the optimization is solved with a coordinate ascent algorithm, iterating be-tween re-analyzing every data point in the data set and re-estimating its hidden structure.”
Learning Deep Structured Models (2014, methodology): “In the future we plan to learn deeper models for object recognition in applications such as holistic semantic scene understanding.”
Variational Inference: A Review for Statisticians (2016, methodology): “In this paper, we review variational inference ( VI ), a method from machine learning for approximating probability densities (Jordan et al., 1999; Wainwright and Jordan, 2008).”
Composing graphical models with neural networks for structured representations and fast inference (2016, background): “Graphical models and exponential families provide many algorithmic tools for efficient inference [15].”
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, background): “, p r (xr|xpa(r), zr) := xr Dr(xpa(r), zr; r) , and rely on variational inference [60], amortised with approximate posteriors q r (zr|xr,xpa(r)) parametrised by encoders in the form of neural nets with parameters r.”
SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements (2017, methodology): “(1999), Wainwright and Jordan (2008)) is an alternative to MCMC for approximating otherwise intractable Bayesian posteriors.”
Kernel Belief Propagation (2011, methodology): “Several theoretical studies have also provided insight into the approximations made by loopy BP, partially justifying its application to graphs with cycles (Wainwright & Jordan, 2008; Yedidia et al., 2001).”
Graphite: Iterative Generative Modeling of Graphs (2018, background): “Using variational arguments [48], we know that the optimal variational marginals assume the following functional form: where N ( i ) denotes the neighbors of variable Z i in the graphical model and O is a function determined by the ﬁxed point equations that depends on the actual potentials.”