Probabilistic latent semantic indexing

By Thomas Hofmann. 1999

Probabilistic Latent Semantic Indexing is a novel approach to automated document indexing which is based on a statistical latent class model for factor analysis of count data. Fitted from a training corpus of text documents by a generalization of the Expectation Maximization algorithm, the utilized model is able to deal with domain{speci c synonymy as well as with polysemous words. In contrast to standard Latent Semantic Indexing (LSI) by Singular Value Decomposition, the probabilistic variant has a solid statistical foundation and de nes a proper generative data model. Retrieval experiments on a number of test collections indicate substantial performance gains over direct term matching methods as well as over LSI. In particular, the combination of models with di erent dimensionalities has proven to be advantageous.

Subfields: Information Retrieval & Search, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Retrieval & Search

In Probabilistic & Bayesian Methods

Citations in the corpus

Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval (2015, neutral): “…NDCG@1 NDCG@3 NDCG@10 TF-IDF model (BM25) 0.305 0.328 0.388 Unigram Language Model (Zhai and Lafferty, 2001) 0.304 0.327 0.385 PLSA(Topic=100) (Hofmann, 1999; Gao et al., 2011) 0.305 0.335 0.402 PLSA(Topic=500) (Hofmann, 1999; Gao et al., 2011) 0.308 0.337 0.402 Latent Dirichlet Allocation…”
From Frequency to Meaning: Vector Space Models of Semantics (2010, neutral): “A low-RAM algorithm, Multislice Projection, for large sparse tensors is presented and evaluated.17 Since the work of Deerwester et al. (1990), subsequent research has discovered many alternative matrix smoothing processes, such as Nonnegative Matrix Factorization (NMF) (Lee & Seung, 1999), Probabili”
Neural Variational Inference for Text Processing (2015, methodology): “Applications of these models on document modelling achieve signiﬁcant improvements on generating documents, compared to conventional probabilistic topic models (Hofmann, 1999; Blei et al., 2003) and also the RBMs (Hinton & Salakhutdinov, 2009; Srivastava et al., 2013).”
A Theory of Multiple-Source Adaptation with Limited Target Labeled Data (2020, methodology): “It is also equivalent to that of a non-negative factorization of the underlying distributions, often adopted in topic modelling or LDA (Hofmann, 2017; Lee and Seung, 2001).”
Discovering objects and their location in images (2005, positive): “When observing a new unseen ‘test’ image, the document specific mixing coefficients P (z|dtest) are computed using the ‘fold-in’ heuristic described in [9].”
Unsupervised Question Answering by Cloze Translation (2019, methodology): “Unsupervised Learning in NLP Most representation learning approaches use latent variables (Hofmann, 1999; Blei et al., 2003), or language (Collobert and Weston, 2008; Mikolov et al., 2013; Pennington et al., 2014; Radford et al., 2018; Devlin et al., 2018).”
Label-Embedding for Image Classification (2015, neutral): “Standard approaches to produce word embeddings from co-ocurrences include Latent Semantic Analyis (LSA) [61], probabilistic Latent Semantic Analysis (pLSA) [62] or Latent Dirichlet Allocation”
Learning to Match using Local and Distributed Representations of Text for Web Search (2016, background): “In contrast, latent semantic analysis (LSA) [5], probabilistic latent semantic analysis (PLSA) [14] and latent Dirichlet allocation (LDA) [2, 40] learn low-dimensional ∗ The author is a part-time PhD student at UCL.”
Document Retrieval Model Through Semantic Linking (2017, background): “Inorder to address the vocabulary gap problem, several researchers such as [12, 3] have already proposed to model documents and queries as a set of words generated from a mixture of latent topics, where a latent topic is a probability distribution over the terms or a cluster of weighted terms, or…”
Multivariate Information Bottleneck (2001, unknown): “A common data type which calls for such a principle is co-occurrence data, such as verbs and direct objects in sentences [7], words and documents [1, 4, 11], tissues and gene expression patterns [14], galaxies and spectral components [10], etc.”
Regression-based latent factor models (2009, unknown): “The factors could be discrete [16, 7, 4] or continuous [22, 17], our work falls in the latter category.”
A Capsule Network for Recommendation and Explaining What You Like and Dislike (2019, background): “n investigated and verified by many works [1 ,3 4 6 8 15 18 20 24 26 27 29 37, 39]. In earlier days, many efforts are made to extract semantic features from reviews with the topic modeling techniques [2, 14]. These works integrate the latent semantic topics into the factor learning models [1 ,18 20 ”
Unsupervised Object Discovery: A Comparison (2010, unknown): “Sivic et al. (2005) have proposed a method that builds on Probabilistic Latent Semantic Analysis (PLSA), as proposed by Hofmann (1999), to separate images of four distinct object categories (faces, airplanes, rear cars, and motorbikes).”
Handling label noise in video classification via multiple instance learning (2011, unknown): “One way to handle label noise is to apply Latent Dirichlet Allocation (LDA) [5], probabilistic Latent Semantic Analysis (pLSA) [14, 23] or its extensions, and remove unrelated latent topics [4, 11].”
Human Action Recognition by Semilatent Topic Models (2009, unknown): “Topic models, such as, LDA, CTM, probabilistic Latent Semantic Analysis (pLSA) [14], and their variants, have been applied to various computer vision applications, such 1762 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 31, NO. 10, OCTOBER 2009 .”
Learning Color Names for Real-World Applications (2009, unknown): “Contrary to this work, we start from the prior-knowledge that color names are ”visual” and that they should be learned from the color distributions (and not for example from texture features), with the aim to improve the quality of the learned color names.”
Learning object categories from Google's image search (2005, unknown): “A leading approach in this field is that of probabilistic Latent Semantic Analysis (pLSA) [12] and its hierarchical Bayesian form, Latent Dirichlet Allocation (LDA) [4].”