A Generalization of Principal Components Analysis to the Exponential Family

By Michael Collins, Sanjoy Dasgupta, Robert E. Schapire. 2001

530 citations recorded by the source corpus.

Subfields: Analytical Chemistry, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Control and Systems Engineering

Citations in the corpus

On the challenges of learning with inference networks on sparse, high-dimensional data (2017, methodology): “Miao et al. learn log-linear models (multinomial-logistic PCA, Collins et al. , 2001) of documents using inference networks.”
Injecting Logical Background Knowledge into Embeddings for Relation Extraction (2015, methodology): “The embeddings can be estimated by maximizing the likelihood of a set of observed ground atoms with `2 regularization (Collins et al., 2001), optimized using stochastic gradient descent.”
Exploiting compositionality to explore a large space of model structures (2012, background): “Exponential family PCA (Collins et al., 2002; Mohamed et al., 2008) generalizes low-rank factorizations to other observation models in the exponential family.”
Variational bounds for mixed-data factor analysis (2010, methodology): “This approach is known as exponential family PCA (ePCA) Graphical Model: [CDS02].”
Variational Extensions to EM and Multinomial PCA (2002, methodology): “A common problem with the earlier formulations of these discrete component analysis [3, 2, 5] is that they fail to make a full probability model of the target data in question, a model where hidden variables, observed data, and assumptions are all clearly exposed.”
Classes and continua of hippocampal CA1 inhibitory neurons revealed by single-cell transcriptomics (2017, unknown): “…Cck interneuron classes have been described, targeting pyramidal cells at multiple locations ranging from their somata to distal dendrites, and the molecular profile of these cells correlates with their synaptic target location (Cope et al., 2002; Klausberger et al., 2005; Somogyi et al., 2004).”
Maximum-Margin Matrix Factorization (2004, background): “It is often desirable, though, to minimize a different loss function: loss corresponding to a specific probabilistic model (where X are the mean parameters, as in pLSA [1], or the natural parameters [2]); or loss functions such as hinge loss appropriate for binary or discrete ordinal data.”
1-Bit Matrix Completion (2012, background): “over the underlying matrix?" Questions of this form are often asked in the context of binary PCA or logistic PCA. There are a number of compelling algorithmic papers on these subjects, including [21,55,36,18,61], which suggest positive answers on simulated and real-world data. In this paper, we”
Generalized Nonnegative Matrix Approximations with Bregman Divergences (2005, background): “[6] discuss algorithms f or obtaining a low rank approximation of the form A ≈ BC, where the loss functions are Bregman divergences, however , there is no restriction on B andC.”
Exponential Family Embeddings (2016, methodology): “Mirror-ing the success of word embeddings, ef-emb models outperform traditional dimension reduction, such as exponential family principal component analysis ( pca ) (Collins et al., 2001) and Poisson factorization (Gopalan et al., 2015), and ﬁnd interpretable features of the data.”
Variational learning for latent Gaussian model of discrete data (2012, methodology): “…analysis for continuous data [Bartholomew et al., 2011; Tipping and Bishop, 1999], binary and multinomial factor analysis for discrete data [Collins et al., 2002; Khan et al., 2010; Mohamed et al., 2008; Wedel and Kamakura, 2001], and Gaussian process regression and classification…”
A Semi-NMF-PCA Unified Framework for Data Clustering (2017, methodology): “The Principal Component Analysis [9], [21] enabled us to ﬁnd the optimal low-dimensional subspace deﬁned by the principal directions Q .”
Iterative Splits of Quadratic Bounds for Scalable Binary Tensor Factorization (2014, methodology): “1 The predictions z t are obtained by the multilinear product of rank-K factors represented in the rows of matrices For matrices ( D = 2 ), this model is a special case of exponential-family PCA [5] where the link function is logistic.”
Poisson Noise Reduction with Non-local PCA (2012, methodology): “The proposed method combines Poisson Principal Component Analysis (Poisson-PCA—a special case of the ExponentialPCA [10, 40]) and sparse Poisson intensity estimation methods [20] in a non-local estimation framework.”
General Latent Feature Models for Heterogeneous Datasets (2017, methodology): “NonGaussian likelihood models have been considered before, for instance, discrete component analysis (Buntine and Jakulin, 2006), exponential family PCA (Collins et al., 2002; Mohamed et al., 2009), exponential family partial least squares (Klami et al.”
Fast maximum margin matrix factorization for collaborative prediction (2005, unknown): “…especially when predicting discrete values such as ratings, loss functions other then sum-squared loss are often more appropriate: loss corresponding to a specific probabilistic model (as in pLSA (Hofmann, 2004) and Exponential-PCA (Collins et al., 2002)) or loss functions such as hinge loss.”
Linear dimensionality reduction: survey, insights, and generalizations (2014, background): “Fifth, another class of popular extensions generalizes PCA to other exponential family distributions, beyond the implicit normal distribution of standard PCA (Collins et al., 2002; Mohamed et al., 2008).”