Supervised Topic Models
By David M. Blei, Jon D. McAuliffe. 2007
1,826 citations recorded by the source corpus.
We introduce supervised latent Dirichlet allocation (sLDA), a statistical model of labelled documents. The model accommodates a variety of response types. We derive a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations. Prediction problems motivate this research: we use the fitted model to predict response values for new documents. We test sLDA on two real-world problems: movie ratings predicted from reviews, and web page popularity predicted from text descriptions. We illustrate the benefits of sLDA versus modern regularized regression, as well as versus an unsupervised LDA analysis followed by a separate regression.
Subfields: Artificial Intelligence, General Social Sciences
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Noah A. Smith
- Quoc V. Le
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
In General Social Sciences
- David M. Blei
- Chengxiang Zhai
- David Mimno
- Hugo Larochelle
- Charu C. Aggarwal
- Jordan L. Boyd-Graber
- Lawrence Carin
- Zhe Gan
Citations in the corpus
- Non-Linear Regression for Bag-of-Words Data via Gaussian Process Latent Variable Set Model (2015, neutral): “Supervised topic model (Blei and McAuliffe 2007) is a topic model of predicting target variables from documents, and uses the low-dimensional vectors for documents as features for prediction.”
- MedLDA: maximum margin supervised topic models (2012, methodology): “In order to consider side information appropriately for discovering more predictive representations, supervised topic models (sL- DA) (Blei and McAuliffe, 2007) introduce a response variable Y to LDA for each document, as shown in Figure 1.”
- Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery (2016, methodology): “, correlated LDA [35], hierarchical Dirichlet processes [36], supervised LDA [37], maximum entropy discriminative LDA [38], geometric LDA [39], and so on.”
- A temporal Bayesian model for classifying, detecting and localizing activities in video sequences (2012, methodology): “One drawback of strictly the LDA-based approaches is the absence of temporal modeling, i.e. restricting the modeling of the correlation of different motion patterns over time.”
- Investigating Capsule Networks with Dynamic Routing for Text Classification (2018, background): “2017). Another common approach is to completely ignore the order of words but focus on their compositions as a collection, whose representatives include probabilistic topic modeling (Blei et al.,2003;Mcauliffe and Blei, 2008) and Earth Mover’s Distance based modeling (Kusner et al.,2015;Ye et al.,20”
- Mining Text Data (2012, methodology): “The second method proposed in [12] is a graph-based approach which directly uses the links in the clustering process.”
- An overview of topic modeling and its current applications in bioinformatics (2016, methodology): “The survLDA model was inspired by sLDA (Mcauliffe and Blei 2008) applied to evaluation of movies, and addressed the following tasks: characterization of cancer subtypes and classification of individual patients according to those subtypes.”
- Recommender Systems (2012, methodology): “LDA was applied to predict review scores based on the content of reviews [194] and to uncover implicit community structures in a social network [195].”
- Mining topics in documents: standing on the shoulders of big data (2014, neutral): “Other types of knowledge, such as document labels have also been used in [3, 29].”
- Proximal Methods for Hierarchical Sparse Coding (2010, positive): “…line of research to pursue could be the supervised design of dictionaries, which has been proved useful in the two frameworks (Mairal et al., 2009a; Bradley and Bagnell, 2009; Blei and McAuliffe, 2008). fact it can be used to obtain approximate solutions of the nonconvex problem presented in Eq.”
- Hallucinated Humans as the Hidden Context for Labeling 3D Scenes (2013, methodology): “As the object affordance is often strongly coupled with the object classes, we use the affordance derived from the learned topics as features that feed into other learning algorithms, similar to the ideas using in supervised topic models [3].”
- fLDA: matrix factorization through latent dirichlet allocation (2010, methodology): “This simultaneous estimation of both user profiles and topic attribution makes our method distinct from recent work called sLDA [7] that incorporates a response variable (like reviews on articles) in deciding LDA topics through a global regression (while our model performs peruser local regression).”
- Differentiable Sparse Coding (2008, methodology): “To match [19] we use vectors of normalized counts for the 5000 words with the highest tf-idf score among the 5006 movie reviews in the data set, use 5-fold cross validation, compute predictions with linear regression on ŵ, and report our performance in terms of predictive R(2) (the fraction of varia”
- Topically Driven Neural Language Model (2017, background): “Both ntm and lda have natural supervised extensions (Cao et al., 2015; McAuliffe and Blei, 2008) for incorporating document labels.”