Probabilistic Latent Semantic Analysis

By Thomas Hofmann. 1999

Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed method is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics. In order to avoid overfitting, we propose a widely applicable generalization of maximum likelihood model fitting by tempered EM. Our approach yields substantial and consistent improvements over Latent Semantic Analysis in a number of experiments.

Subfields: Natural Language Processing, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Probabilistic & Bayesian Methods

Citations in the corpus

Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions (2011, methodology): “Then probabilistic latent semantic analysis (pLSA) by Hofmann (1999) is applied to embed the images into a latent semantic space to get low-dimensional representations called topic distributions (here 20 topics are used, and thus each image is converted to a 20 -dimensional probability vector).”
Multi-Relational Latent Semantic Analysis (2013, background): “…Siamese neural networks (Yih et al., 2011), recurrent neural networks (Mikolov et al., 2013) and recursive neural networks (Socher et al., 2011), or by graphical models such as probabilistic latent semantic analysis (PLSA) (Hofmann, 1999) and latent Dirichlet allocation (LDA) (Blei et al., 2003).”
A Comprehensive Survey on Transfer Learning (2019, background): “CD-PLSA is an extension of dual-PLSA.”
Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval (2015, methodology): “PLSA (Probabilistic Latent Semantic Analysis) is a topic model proposed in [37], which is trained using the Maximum A Posterior estimation [38] on the documents side from the same training dataset.”
Probabilistic latent semantic indexing (1999, neutral): “The same mechanism is able to detect \true" polysems [6].”
Topic-aware social influence propagation models (2012, unknown): “Since this test is time-independent, we also use as a baseline for comparison the Probabilistic Latent Semantic Analysis (pLSA) model [15].”
Expectation-Propogation for the Generative Aspect Model (2002, methodology): “Hofmann’s original aspect model involved a large number of parameters and heuristic procedures to avoid overfitting (Hofmann, 1999). Blei et al. (2001) introduced a modified model with a proper generative semantics and used variational methods to carry out inference and learning.”
Probabilistic topic models (2012, unknown): “• Builds on latent semantic analysis (Deerwester et al., 1990; Hofmann, 1999) It is mixed membership model (Erosheva, 2004).”
Unsupervised learning of auditory filter banks using non-negative matrix factorisation (2008, unknown): “In [5] it is proven that a point which is invariant under these up-date rules is also invariant under the update formulas of the algorithm proposed in [6] for probabilistic latent semantic analysis (PLSA).”
Web page classification: Features and algorithms (2009, background): “, probabilistic (Hofmann 1999b; Hofmann 1999a)) and make it more applicable to large datasets.”
Audio/video fusion for objects recognition (2009, background): “From a theoretical point of view a large part of the existing merging methods involve mutual information between modalities [13], [11] or are based on statistical techniques [14], [7].”
Target-Dependent Sentiment Classification With BERT (2019, background): “pLSI [10] and LDA [3] relax the independence assumption by introducing an aspect or topic, but the mapping does not simplify the original vector which is still in vocabulary size V.”
Co-regularized Multi-view Spectral Clustering (2011, unknown): “Since spectral clustering essentially works with similarities of the data, we ﬁrst project the data using Latent Semantic Analysis (LSA) [16] to a 100 -dimensional space and compute similarities in this lower dimensional space.”
The rate adapting poisson model for information retrieval and object recognition (2006, methodology): “For a query document q, P (z|q) was computed using 25 iterations of the folding-in heuristi c (Hofmann, 1999).”