Quickly Generating Representative Samples from an RBM-Derived Process

By Olivier Breuleux, Yoshua Bengio, Pascal Vincent. 2011

80 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Generative Adversarial Nets (2014, background): “[7] and used for various generative models for which the exact likelihood is not tractable [24, 3, 4].”
Better Mixing via Deep Representations (2012, methodology): “In addition, we measure the quality of the obtained samples, using a procedure for the comparison of sample generators described in Breuleux et al. (2011).”
A Generative Process for Contractive Auto-Encoders (2012, methodology): “To get a more objective quantitative measure of the quality of the samples, we resort to a procedure proposed in Breuleux et al. (2011) that can be applied to compare arbitrary sample generators.”
Learning to Generate Samples from Noise through Infusion Training (2017, methodology): “Since most previous published results on non-likelihood based models (such as GANs) used a Parzen-window-based estimator (Breuleux et al., 2011), we use it as our ﬁrst comparison tool, even if it can be misleading (Lucas Theis & Bethge, 2016).”
GSNs : Generative Stochastic Networks (2015, methodology): “Following Breuleux et al. (2011), the quality of the samples was also estimated quantitatively by measuring the log-likelihood of the test set under a Parzen density estimator constructed from 10,000 consecutively generated samples (using the realvalued mean-field reconstructions as the training dat”
Generative adversarial networks (2014, neutral): “inst other models of the real-valued (rather than binary) version of dataset. of the Gaussians was obtained by cross validation on the validation set. This procedure was introduced in Breuleux et al. [8] and used for various generative models for which the exact likelihood is not tractable [25, 3, 5”
Representation Learning: A Review and New Perspectives (2012, methodology): “Tieleman and Hinton [205] have proposed quite a different approach to addressing potential mixing problems of SML with their FPCD, and it has also been exploited to train DBMs [173] and construct a pure sampling algorithm for RBMs [39].”
Autoencoding beyond pixels using a learned similarity metric (2015, methodology): “Because the log-likelihood is intractable, it is often estimated using a Parzen window approach (Breuleux et al., 2011), but even for small images of 6×6 pixels this is shown to be biased in favor of methods that overfit (Theis et al.”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “An interesting alternative is rates-FPCD (Tieleman and Hinton, 2009; Breuleux et al. , 2011) which appears to be more robust to this problem and generally mixes faster, but at the cost of losing theoretical guarantees.”
Discovering Hidden Factors of Variation in Deep Networks (2014, methodology): “eLU 240-5 Maxout 2000 ReLU 500 ReLU 240-5 Maxout 2000 ReLU 784 Linear 784 Linear 2304 Linear GENERATIVE PERFORMACE Table 5 shows Parzen-window based likelihood estimates from our models introduced in Breuleux et al. (2011). We use the code and results from Goodfellow et al. (2014). For our model, we”
Cooperative Training of Descriptor and Generator Networks (2016, methodology): “This evaluation method was proposed by [61] and has been used by”
Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture (2018, unknown): “MNIST data set [51].”
On the Expressive Power of Deep Architectures (2011, background): “Theory suggests (Younes, 1999) this is a good estimator if the parameter changes are small, but practice revealed (Tieleman, 2008) that it worked even for large updates, in fact giving rise to faster mixing (Tieleman and Hinton, 2009; Breuleux et al., 2011).”
Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy (2016, methodology): “An approach used in several studies is to obtain a Parzen window estimate of the density and compute the log-likelhiood (Goodfellow et al., 2014; Nowozin et al., 2016; Breuleux et al., 2011).”
A Review on Deep Learning Techniques for Video Prediction (2020, background): “Some other alternative metrics include the inception score [232] introduced to deal with GANs mode collapse problem by measuring the diversity of generated samples; perceptual similarity metrics, such as DeePSiM [44]; measuring sharpness based on difference of gradients [43]; Parzen window [233], ye”
Unsupervised Learning of Visual Structure using Predictive Generative Networks (2015, methodology): “…from [ − π/ 2 , π/ 2] , where 0 corresponds to a frontal Generative models are often evaluated using a Parzen window estimate of the log-likelihood (Breuleux et al., 2011), but due to the deﬁciencies of this approach for high dimensional images, we chose values of the weighting parameter…”