Advanced mean field methods: theory and practice

By Manfred Opper, David Saad. 2001

564 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b2676e79e9c9817397ea6a75dedf902b86d1d5bc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Bayesian reasoning and machine learning (2012, unknown): “which is called the ‘free energy’ bound in the physics community[236].”
Learning message-passing inference machines for structured prediction (2011, methodology): “The classical “mean-field” (and more generally variational methods [15]) method is easily framed as a simpler message passing strategy where, instead of”
Advances in Variational Inference (2017, methodology): “community and further developed by the machine learning community [143], [150], [190].”
A CDMA multiuser detection algorithm on the basis of belief propagation (2003, methodology): “Recent research has revealed that this algorithm is closely related to the transfer matrix method and the Bethe approximation [10, 11], which are standard techniques in statistical mechanics, and exhibits excellent performance when a given statistical model is expressed by a sparse graph [12–14].”
The max-min hill-climbing Bayesian network structure learning algorithm (2006, methodology): “Variational methods for learning the graphical model structure have mainly been applied to restrictive models (Ghahramani & Beal, 2001), run on very small networks (Friedman, 1998), or used to develop scoring functions (Ghahramani & Beal, 2003), and therefore are not included in this study.”
Variational inference for Dirichlet process mixtures (2006, methodology): “One class of alternatives is provided by variational methods, a class of deterministic algorithms that convert inference problems into optimization problems (Opper and Saad, 2001; Wainwright and Jordan, 2003).”
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm (2019, methodology): “The Teacher-Student paradigm has been broadly used to study supervised learning (Saad and Solla, 1995; Monasson and Zecchina, 1995; Opper and Saad, 2001; Engel and Van den Broeck, 2001; Zdeborová and Krzakala, 2016; Barbier et al., 2019; Gabrié et al., 2018; Aubin et al., 2018; Franz et al., 2018).”
Stochastic Gradient Descent as Approximate Bayesian Inference (2017, methodology): “This is done by minimizing Kullback-Leibler divergence or related divergences between these distributions (Jordan et al., 1999; Opper and Saad, 2001).”
Statistical Physics of Spin Glasses and Information Processing (2001, background): “In Opper and Saad (2001), one finds an extensive coverage of recent developments in applications of meanfield theory to interdisciplinary fields as well as a detailed elucidation of various aspects of mean-field theory.”
The IM algorithm: a variational approach to Information Maximization (2003, methodology): “More successful decoding algorithms approximate the posterior mean[10].”
Probabilistic Matrix Factorization with Non-random Missing Data (2014, methodology): “Therefore we approximate this distribution using expectation propagation (EP) (Minka, 2001) and variational Bayes (VB) (Ghahramani & Beal, 2001).”
Expectation Consistent Approximate Inference (2005, background): “In this section we derive the EC free energy for a specific model, the quadratic, and discuss several possible choices for the consistent statistics 〈g(x)〉.”
A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors (2002, background): “Although the NP-hardness proved by Verdu [51] seems unavoidable, we expect that the mean-field approximation [ 52 ], a notion in statistical mechanics to systematically approximate microscopic behavior of systems, can provide practical means of implementing the MPM detectors approximately.”
Inverse statistical problems: from the inverse Ising problem to data science (2017, background): “However, there are many approximation schemes for F (J ,h) [165], which lead to approximations for the entropy and the Gibbs free energy.”