Consistency of Pseudolikelihood Estimation of Fully Visible Boltzmann Machines

By Aapo Hyvärinen. 2006

108 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

GSNs : Generative Stochastic Networks (2015, neutral): “These results also show in a novel way (see e.g. Hyvärinen (2006) for earlier results) that training by pseudolikelihood or generalized pseudolikelihood provides a consistent estimator of the associated joint, so long as the GSN Markov chain defined above is ergodic.”
How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? (2015, background): “The model likelihood is often considered the most principled training objective and most research in the past decades has focussed on maximum lieklihood(ML) and approximations thereof Hinton et al. (2006); Hyv¨arinen (2006); Kingma & Welling (2013).”
Unsupervised Generative Modeling Using Matrix Product States (2017, methodology): “Second, with MPS, we can monitor the training process easily using exact NLL instead of other quantities such as reconstruction error or pseudolikelihood for RBM, which introduce bias to monitoring [67].”
Bayesian reasoning and machine learning (2012, unknown): “At least for the case of the Boltzmann machine, this forms a consistent estimator[139].”
A unified approach to building hybrid recommender systems (2009, methodology): “Because of this, we approximate the log likelihood of the training set by its log pseudo-likelihood [3, 14]: X”
Inverse statistical problems: from the inverse Ising problem to data science (2017, background): “Pseudolikelihood is an alternative to the likelihood function (18), which leads to the exact inference of model parameters in the limit of an inﬁnite number of samples [3, 96, 142].”
Connections Between Score Matching, Contrastive Divergence, and Pseudolikelihood for Continuous-Valued Variables (2007, unknown): “This was shown in [6] for a fully-visible Boltzmann machine, with CD based on Gibbs sampling.”
Some extensions of score matching (2007, methodology): “which has the benefit that it shows the close connection to pseudo-likelihood derived in Hyvarinen (2006) .”
Gradient Estimators for Implicit Models (2017, methodology): “As a parameter estimation technique, score matching is also related to contrastive divergence [12], pseudo likelihood estimation [15], and denoising auto-encoders [46].”