Products of Experts

By Max Welling Donald. 2007

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Generalized Denoising Auto-Encoders as Generative Models (2013, methodology): “The spirit of this procedure is thus very similar to the CD-k (Contrastive Divergence with k MCMC steps) procedure proposed to train RBMs (Hinton, 1999; Hinton et al. , 2006).”
GSNs : Generative Stochastic Networks (2015, methodology): “The spirit of this procedure is thus very similar to the CD-k (Contrastive Divergence with k MCMC steps) procedure proposed to train RBMs (Hinton, 1999; Hinton et al., 2006).”
Representation Learning: A Review and New Perspectives (2012, methodology): “Contrastive Divergence (Hinton, 1999; Hinton et al., 2006), Stochastic Maximum Likelihood (Younes, 1999; Tieleman, 2008) and fast-weights persistent contrastive divergence or FPCD (Tieleman and Hinton, 2009) are all ways to avoid or reduce the need for burn-in.”
Learning Factored Representations in a Deep Mixture of Experts (2013, background): “A product of experts (PoE) [6] is similar, but instead combines log probabilities to form a product: Also closely related to our work is the Hierarchical Mixture of Experts [9], which learns a hierarchy of gating networks in a tree structure.”
Learning Distributed Representations from Reviews for Collaborative Filtering (2015, background): “As argued in [8], a product of experts can more easily model a peaky distribution, especially, in a high-dimensional space.”
Deep Energy Estimator Networks (2018, background): “PoE is powerful in partitioning the space in a distributed representation formed by ε α ( x ; θ ) , in contrast to mixture models that partition the space with one region per expert (Hinton, 1999).”
Implicit Generation and Generalization in Energy-Based Models (2019, background): “Assuming independence between each conditional distribution, generation through joint conditioning on all latents is represented by generation through an energy function represented as the sum of each latent conditional energy function (Hinton, 1999) and corresponds to a product of experts model.”
Fields of Experts (2009, methodology): “We propose to represent the MRF potentials as a Product of Experts (Hinton 1999) with the same basic form as in (7).”
Product of Experts for Statistical Parametric Speech Synthesis (2012, background): “A product of experts (PoE) [9], [33] combines multiple models (experts) by taking their product and normalizing the result.”
Structured learning with constrained conditional models (2012, methodology): “(3)), we adopt the idea of “product of experts” (Hinton 1999), where the HMM is the expert that predicts the probability of the label assignment, and the constraints component downgrades solutions that violate the constraints.”
Parallel Tempering for Training of Restricted Boltzmann Machines (2010, methodology): “The most popular learning algorithm for RBMs is the Contrastive Divergence (CD) algorithm (Hinton, 1999; Hinton, 2002).”
Compositional Visual Generation with Energy Based Models (2020, methodology): “In prior work, [10] show that products of EBMs can be used to decompose complex generative modeling problems to simpler ones.”
Learning to Learn Semantic Parsers from Natural Language Supervision (2019, background): “Note that this model is closely related to co-training (Blum and Mitchell, 1998) (and more generally multi-view learning (Xu et al., 2013)) and is also a special case of a product-of-experts (PoE) model (Hinton, 1999).”
Forward Attention in Sequence- To-Sequence Acoustic Modeling for Speech Synthesis (2018, methodology): “The method of forward attention with transition agent can also be explained from the point of view of a product-of-experts model (PoE) [24, 25].”
A Game-Theoretic Approach to Generating Spatial Descriptions (2010, background): “• If w is rooted at NP, we recursively compute the distributions over objects g for each child tree, multiply the probabilities, and renormalize (Hinton, 1999).”
Representational Power of Restricted Boltzmann Machines and Deep Belief Networks (2008, background): “something about the representational power of Rn as increases? Denoting Dn the set of distributions one can obtain with R n , it follows from the unfolding argument in Hinton et al. (2006) that Dn ⊆ Dn +1.”
Context-Based Morphological Disambiguation with Random Fields (2005, background): “Recent work in learning weights for the component “expert” models has turned to cooperativetechniques (Hinton, 1999).”