Rethinking LDA: Why Priors Matter

By Hanna M. Wallach, David Mimno, Andrew McCallum. 2009

775 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Rethinking LDA: Moment Matching for Discrete ICA (2015, methodology): “e whitening matrix. Variational inference. For variational inference, we used the code of D. Blei and modiﬁed it for the estimation of a non-symmetric Dirichlet priorc, which is known to be important [35]. The default values of the tolerance/maximumnumber of iterations parameters are used for variat”
Stochastic variational inference (2012, background): “We note that Blei et al. (2003) and Wallach et al. (2009) found improved empirical performance with non-exchangeable priors.”
A Thousand Frames in Just a Few Words: Lingual Description of Videos through Latent Topics and Sparse Object Stitching (2013, positive): “First, we use an asymmetric Dirichlet prior, α for the document level topic proportions θd following [31] unlike the symmetric one in [4].”
Online Learning for Latent Dirichlet Allocation (2010, background): “For simplicity, we assume symmetric priors on θ and β, but this assumption is easy to relax [8].”
Latent Variable Models of Selectional Preference (2010, methodology): “However, Wallach et al. (2009) demonstrate that LDA is relatively insensitive to larger-than-necessary choices of Z when the Dirichlet parameters α are optimised as part of model estimation.”
Understanding the Limiting Factors of Topic Modeling via Posterior Contraction Analysis (2014, background): “Meanwhile, Wallach et al. (2009) studied the role of asymmetric Dirichlet priors.”
Aspect Extraction through Semi-Supervised Modeling (2012, background): “Asymmetric Beta priors: Hyper-parameters α, β, β are not very sensitive and the heuristic values suggested in (Griffiths and Steyvers, 2004) usually hold well in practice (Wallach et al. 2009).”
An architecture for parallel topic models (2010, background): “While this design choice is quite unrealistic it turns out not to matter significantly in practice [8].”
Simultaneously Discovering and Quantifying Risk Types from Textual Risk Disclosures (2014, result): “There are many discussions on the advantages and disadvantages of these two types of learning algorithms, and some previous studies (Teh et al. 2007, Asuncion et al. 2009, Wallach et al. 2009, Zhai et al. 2012) have attempted to compare their performance.”
Supervised Topic Models (2007, methodology): “While we ﬁx the Dirichlet parameter α to 1 /K in Section 4, other applications of topic modeling ﬁt this parameter (Blei et al., 2003; Wallach et al., 2009).”
How to marry a star: Probabilistic constraints for meaning in context (2020, background): “Finally, the “sense frequency” row shows the result of assuming asymmetric prior probabilities for the scenarios (Wallach et al. 2009), which translate into prior probabilities for the senses of argument .”
Weakly Supervised Joint Sentiment-Topic Detection from Text (2012, neutral): “substantial advantage over a symmetric prior [25].”
Interactive Topic Modeling (2011, background): “Normally, estimating hyperparameters is important for topic modeling (Wallach et al., 2009).”
Learning Tags from Unsegmented Videos of Multiple Human Actions (2011, methodology): “However, we emphasize that it is not important to tune, because good results can always obtained by setting a large number of topics and allowing hyperparameter learning of vector α to reduce the weight of unused topics [25].”
Learning from Your Network of Friends: A Trajectory Representation Learning Model Based on Online Social Ties (2016, methodology): “We adopt two baseline models for comparisons: LDA [4], and LDA with asymmetric location-activity hyperprior (LDA-A) [8].”
Model-Based Purchase Predictions for Large Assortments (2016, methodology): “This extends the text modeling implementation of LDA, where these probabilities are typically considered to be known, or at best calibrated using heuristics (Wallach et al., 2009; Asuncion et al., 2009).”
WarpLDA: a Cache Efficient O(1) Algorithm for Latent Dirichlet Allocation (2015, background): “ WarpLDA by the careful choice of proposal distributions qdoc(z dn= k) /C :dn dk + qword(z dn= k) /C :dn wk + w: (7) Here, we assume symmetric Dirichlet prior, i.e., 8k; k = . Although Wallach et al. [24] show that using asymmetric prior can lead to better perplexity results, it is non-trivial to le”