The EM algorithm for mixtures of factor analyzers

By Zoubin Ghahramani, Geoffrey E. Hinton. 1996

Subfields: Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Neural Autoregressive Distribution Estimation (2016, methodology): “alidation run. Early stopping was important to avoid overtting the RNADE models. It also improved the results of the MFAs, but to a lesser degree. The MFA models were trained using the EM algorithm (Ghahramani and Hinton, 1996; Verbeek, 2005). We cross-validated the number of components and factors.”
RNADE: The real-valued neural autoregressive density-estimator (2013, methodology): “The MFA models were trained using the EM algorithm [24, 25], the number of components and factors were crossvalidated.”
Deep Mixtures of Factor Analysers (2012, methodology): “To deal with non-linear data distributions, Mixtures of Factor Analyzers (MFA) (Ghahramani & Hinton, 1996) can be used.”
Probabilistic Linear Discriminant Analysis for Inferences About Identity (2007, methodology): “To learn the MixPLDA model we apply the standard recipe for learning mixtures of distributions (e.g. See [ 7 ] and [4] Chap.”
Variational Learning in Nonlinear Gaussian Belief Networks (1999, methodology): “Once this distribution has been computed it is straightforward to adjust the generative weights to maximize the likelihood of the observed data using either a gradient method or the expectation maximization (EM) algorithm (Ghahramani and Hinton 1996).”
EM Algorithms for PCA and SPCA (1997, methodology): “The number of floating point operations required to find the first principal component was recorded using MATLAB’s flops function.”
Continuous latent variable models for dimensionality reduction and sequential data reconstruction (2001, methodology): “Again, a maximum likelihood estimate for the parameters of the model was found via an EM algorithm13 (Ghahramani and Hinton, 1996), with random starting point and stopping when the relative increment in log-likelihood was smaller than 10−4.”
A tutorial on task-parameterized movement learning and retrieval (2015, methodology): “(cid:17) (56) As highlighted by [32], the same process can be used to estimate the second moment of the factors E (cid:16) z ( j ) z ( j ) ⊤ | X ( j ) (cid:17) , which provides a measure of uncertainty in the factors that has no analogue in PCA.”
A Bayesian approach for place recognition (2012, methodology): “Mixture of linear models are similar to mixtures of factor analysers, that are commonly used to perform simultaneous clustering and local dimensionality reduction [9].”
Interpretable and Differentially Private Predictions (2019, methodology): “Mixtures of factor analyzers (MFA) are also similar to ME models, but for density estimation of high-dimensional realvalued data (Ghahramani and Hinton 1997)”
Unsupervised image classification, segmentation, and enhancement using ICA mixture models (2002, methodology): “mixture model is also similar to the mixtures of factor analyzers proposed by Ghahramani and Hinton [19].”
A Unifying Review of Linear Gaussian Models (1999, methodology): “…been considered: mixtures of linear dynamical systems are known as switching state-space models (see Shumway & Stoffer, 1991; Ghahramani & Hinton, 1996b); mixtures of factor analyzers (Ghahramani and Hinton, 1997) and of pancakes (PCA) (Hin-ton et al., 1995); and mixtures of HMMs (Smyth, 1997).”
Face Detection: A Survey (2001, methodology): “Given a set of training images, the EM algorithm [34] is used to estimate the parameters in the mixture model of factor analyzers [49].”
On GANs and GMMs (2018, methodology): “Rather than using EM [22, 11] (which is problematic with large datasets) we decided to optimize the log-likelihood (equation 2) using Stochastic Gradient Descent and utilize available differentiable programming frameworks [1] that perform the optimization on GPU.”