Learning in Markov Random Fields with Contrastive Free Energies

38 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Statistical inference using weak chaos and infinite memory (2010, neutral): “For these models we may write, KL[P0||Pw] = < E(S;w) >P0 + log(Z) − H[P0] (12) = (< E(S;w) >P0 − < E(S;w) >Pw) − (H[P0] − H[Pw]) (13) with E(S;w) = −( ∑ ij wijsisj + ∑ i wisi) (14) This represents a difference of two free energies, ∆F or “contrastive free energy” (Welling & Sutton, 2005).”
Learning Algorithms for the Classification Restricted Boltzmann Machine (2012, methodology): “Keywords: restricted Boltzmann machine, classification, discriminative learning, generative learning”
Learning in Markov Random Fields using Tempered Transitions (2009, background): “When modeling high-dimensional, richly structured data, the inference problem becomes much more difficult because the distribution we need to infer is likely to be highly multimodal [17].”
Fields of Experts (2009, background): “…et al., 1997), score matching (Hyvaärinen, 2005), discriminative training of energy-based models (LeCun and Huang, 2005), as well as a large set of variational (and related) approximations to maximum likelihood (Jordan et al., 1999; Yedidia et al., 2003; Welling and Sutton, 2005; Minka, 2005).”
A Better Way to Pretrain Deep Boltzmann Machines (2012, background): “Multiple layers of hidden units make learning in DBM’s far more difficult [13].”