Probabilistic Principal Component Analysis

By Michael E. Tipping, Charles M. Bishop. 1999

Subfields: Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Beyond CCA: Moment Matching for Multi-View Models (2016, neutral): “Gaussian CCA is subject to some well-known unidentiﬁ-ability issues, in the same way as the closely related factor analysis model (FA; Bartholomew, 1987; Basilevsky, 1994) and its special case, the probabilistic principal component analysis model (PPCA; Tipping & Bishop, 1999; Roweis, 1998).”
Rethinking LDA: Moment Matching for Discrete ICA (2015, positive): “The LDA model (2) can be seen as a discretization of principal component analysis (PCA) via replacement of the normal likelihood with the multinomial one and adjusting the prior [9] in the following probabilistic PCA model [10, 11]: θ ∼ Normal(0, IK) and x|θ ∼ Normal(Dθ, σ(2)IM ), where D ∈ RM×K is ”
Representation Learning: A Review and New Perspectives (2012, methodology): “PCA can be given a natural probabilistic interpretation (Roweis, 1997; Tipping and Bishop, 1999) as factor analysis: p(h) = N (h; 0, σ2hI) p(x | h) = N (x;Wh+ µx, σ2xI), (1) where x ∈ Rdx , h ∈ Rdh , N (v;µ,Σ) is the multivariate normal density of v with mean µ and covariance Σ, and columns of W…”
Stochastic variational inference (2012, neutral): “…models (McCullagh and Nelder, 1989; Girolami and Rogers, 2006), probabilistic factor analysis/matrix factorization models (Spearman, 1904; Tipping and Bishop, 1999; Collins et al., 2002; Wang, 2006; Salakhutdinov and Mnih, 2008; Paisley and Carin, 2009; Hoffman et al., 2010b), certain…”
Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders (2019, methodology): “Due to the fact that we are dealing with time-varying treatment assignments, we cannot use standard factor models, such as Principal Component Analysis (Tipping & Bishop, 1999) or Deep Exponential Families (Ranganath et al., 2015), as they can only be applied in the static setting.”
Theory and Evaluation Metrics for Learning Disentangled Representations (2019, methodology): “Standard VAE is theoretically similar to probabilistic PCA (pPCA) [41] as both assume the same generative process p ( x, z ) = p θ ( x | z ) p ( z ) .”
High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes (2019, methodology): “Deep learning-based sequence models have been successfully applied to audio signals [33], language modeling [13, 30], and general density estimation of univariate sequences [22, 21].”
The Variational Gaussian Process (2015, background): “We also recover a form of factor analysis (Tipping & Bishop, 1999) in the variational space.”
Bayesian group factor analysis with structured sparsity (2016, methodology): “With an isotropic noise assumption, Σ = Iσ 2 , this model has a probabilistic principal components analysis interpretation (Roweis, 1998; Tipping and Bishop, 1999b).”
From learning models of natural image patches to whole image restoration (2011, background): “Interpreted as a probabilistic model [24] PCA can be regarded as a Gaussian model, taking into account only the first and second moments of the data.”
DEEProtect: Enabling Inference-based Access Control on Mobile Sensing Applications (2017, methodology): “The principal component analysis (PCA) technique [84] can also be utilized to reveal hidden structure in mobile sensor data.”
Collaborative Filtering: A Machine Learning Perspective (2004, unknown): “8 where x is an observed data vector, z is a vector in the latent space, Λ an N ×K matrix which maps the K dimensional latent space vectors into the N dimensional data space, μ specifies a mean in the data space common to all data vectors, and is randomly sampled Gaussian noise unique to each data v”
Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming (2005, background): “1 MLE via L1 norm minimization The subspace estimation problem can be formulated as a maximum likelihood estimation (MLE) problem [23].”
Generalized principal component analysis (GPCA) (2005, background): “In Probabilistic PCA [20] (PPCA), the noise is assumed to be drawn from an unknown distribution and the problem becomes one of identifying the subspace and distribution parameters in a maximum-likelihood sense.”
EM Algorithms for PCA and SPCA (1997, background): “Furthermore, Tipping and Bishop have shown [8, 9] that the only stable local extremum is the global maximumat which the true principal subspace is found; so it converges to the correct resul .”
Bayesian reasoning and machine learning (2012, unknown): “4 Probabilistic Principal Components Analysis PPCA[289] corresponds to Factor Analysis under the restriction Ψ = σ(2)ID.”
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (2017, methodology): “Given these candidates for cluster centers, we assign each aligned point to a cluster representing its nearest candidate and then run the EM algorithm of [38], building a mixture of probabilistic PCA bases.”