EM Algorithms for PCA and SPCA

By Sam T. Roweis. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Beyond CCA: Moment Matching for Multi-View Models (2016, neutral): “Gaussian CCA is subject to some well-known unidentifiability issues, in the same way as the closely related factor analysis model (FA; Bartholomew, 1987; Basilevsky, 1994) and its special case, the probabilistic principal component analysis model (PPCA; Tipping & Bishop, 1999; Roweis, 1998).”
Rethinking LDA: Moment Matching for Discrete ICA (2015, positive): “The LDA model (2) can be seen as a discretization of principal component analysis (PCA) via replacement of the normal likelihood with the multinomial one and adjusting the prior [9] in the following probabilistic PCA model [10, 11]: θ ∼ Normal(0, IK) and x|θ ∼ Normal(Dθ, σ(2)IM ), where D ∈ RM×K is ”
Bayesian group factor analysis with structured sparsity (2016, methodology): “With an isotropic noise assumption, Σ = Iσ2, this model has a probabilistic principal components analysis interpretation (Roweis, 1998; Tipping and Bishop, 1999b).”
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions (2009, methodology): “See [80, 87] for an analysis of Krylov-subspace methods with a random start and [111, 113] for work on the randomized power scheme.”
Pattern Recognition and Machine Learning (2006, unknown): “One of the benefits of the EM algorithm for PCA is computational efficiency for large-scale applications (Roweis, 1998).”
Face transfer with multilinear models (2005, methodology): “, PPCA [Tipping and Bishop 1999], SPCA [Roweis 1997], or ISVD [Brand 2002]).”
Random projection in dimensionality reduction: applications to image and text data (2001, methodology): “There exists computationally less expensive methods [26, 24] for finding only a few eigenvectors and eigenvalues of a large matrix; in our experiments, we use appropriate Matlab routines to realize these.”
General Latent Feature Models for Heterogeneous Datasets (2017, methodology): “Some examples include principal component analysis (PCA) (Pearson, 1901), probabilistic PCA (Tipping and Bishop, 1997; Roweis, 1997), independent component analysis (Hyvärinen, 1997) and factor analysis (Thurstone, 1931), among others.”
Incremental Learning for Robust Visual Tracking (2008, background): “It can be shown (Roweis 1997) that the negative exponential distance from It to the subspace spanned by U , i.”
Unsupervised Learning (2018, unknown): “Linear (e.g., principal components analysis, PCA) and nonlinear (e.g., multidimensional scaling, MDS, or Sam-mon’ mapping) projection methods can be used as meth-ods of projection of the input vectors on a reduced space by preserving the variance of the data or the dissimilarities among the input ve”
Realtime facial animation with on-the-fly correctives (2013, methodology): “In particular, [Roweis 1998] proves that the iterative estimations of MK converge to the true PCA solution using the EM algorithm.”
Integrating model simulation tools and cryo‐electron microscopy (2022, methodology): “…identify classes from a continuously variable population of particles 62 : here, probabilistic principal component analysis (where principle components are estimated using an expectation maximization algorithm) 67,68 is used to identify the main structural fluctuations within the imaged molecules.”
A tutorial on adaptive MCMC (2008, methodology): “Roweis (1997) also provides an EM-algorithm for PPCA without taking the zero-error limit, in what is called Sensible PCA.”
Advanced Lectures on Machine Learning (2004, methodology): “The most commonly used technique for feature reduction is principal component analysis (PCA), also known as the Karhunen-Loeve Transform (KLT).”
An Introduction to Variational Autoencoders (2019, methodology): “In (Roweis, 1998) it was shown that PCA corresponds to the maximum-likelihood (ML) solution of a special case of the linear-Gaussian model with a prior p(z) = N (0, I) and a conditional distribution p(x|z) = N (x; Wz, I), specifically the case with infinitesimally small .”