Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora

By Daniel Ramage, David Leo Wright Hall, Ramesh Nallapati, Christopher D. Manning. Conference on Empirical Methods in Natural Language Processing, 2009

1,475 citations recorded by the source corpus.

A significant portion of the world's text is tagged by readers on social bookmarking websites. Credit attribution is an inherent problem in these corpora because most pages have multiple tags, but the tags do not always apply with equal specificity across the whole document. Solving the credit attribution problem requires associating each word in a document with the most appropriate tags and vice versa. This paper introduces Labeled LDA, a topic model that constrains Latent Dirichlet Allocation by defining a one-to-one correspondence between LDA's latent topics and user tags. This allows Labeled LDA to directly learn word-tag correspondences. We demonstrate Labeled LDA's improved expressiveness over traditional LDA with visualizations of a corpus of tagged web pages from del.icio.us. Labeled LDA outperforms SVMs by more than 3 to 1 when extracting tag-specific document snippets. As a multi-label text classifier, our model is competitive with a discriminative baseline on a variety of datasets.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/e9a822d6fe66b0cfd0b4c5a10411172b80346bf1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Exploiting Social Relations and Sentiment for Stock Prediction (2014, methodology): “Ramage et al. (2009; 2011) presented a partially supervised learning model called Labeled LDA to utilize supervision signal in topic modeling.”
Extracting Diurnal Patterns of Real World Activity from Social Media (2013, background): “Several Probabilistic Topic Models were proposed for thematic analysis of Twitter messages (Zhao et al. 2011; Hu et al. 2012; Ramage et al. 2009).”
MedLDA: maximum margin supervised topic models (2012, background): “More variants of supervised topic models can be found in a number of applied domains, such as the aspect rating model (Titov and McDonald, 2008) for predicting ratings for each aspect of a hotel and the credit attribution model (Ramage et al., 2009) that associates each word with a label.”
Characterizing Microblogs with Topic Models (2010, methodology): “We use a generalization of LDA, Labeled LDA (Ramage, et al., 2009), which extends LDA by incorporating supervision where available.”
A Model of Text for Experimentation in the Social Sciences (2016, methodology): “…to fully automated approaches, work on interactive topic modeling and user-specified constraints is particularly appropriate to social scientists who may have a deep knowledge of their particular document sets (Ramage et al. 2009; Andrzejewski et al. 2011; Hu, BoydGraber, and Satinoff 2011).”
An overview of topic modeling and its current applications in bioinformatics (2016, background): “Hierarchical labeled LDA (HLLDA) (Petinot et al. 2011) is a Bayesian model that introduced a label prior into hLDA.”
DEMix Layers: Disentangling Domains for Modular Language Modeling (2021, methodology): “Incorporating Metadata Document metadata has been commonly used to improve the quality of topic models (Mimno and McCallum, 2012; Ramage et al., 2009; Zhu et al., 2012), and previous works have used metadata for adapting RNN-based language models (Jaech and Ostendorf, 2018) or learning better docume”
Classifying Political Orientation on Twitter: It's Not Easy! (2013, methodology): “As part of the present study we employed a Labeled LDA as a stand-alone classifier and found it to perform as well as SVMs, suggesting that it might be fruitfully used as primary attribute inference system (Ramage et al. 2009).”
Interactive Topic Modeling (2011, methodology): “, 2010); this topic shouldn’t be associated with this document but is (Ramage et al., 2009); these words shouldn’t be the in same topic but are (Andrzejewski et al.”
Mining topics in documents: standing on the shoulders of big data (2014, neutral): “Other types of knowledge, such as document labels have also been used in [3, 29].”
Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey (2017, methodology): “Labeled LDA (LLDA) is another LDA extension which suppose that each document has a set of known labels [124].”
Incorporating Lexical Priors into Topic Models (2012, methodology): “Of these models, the most related one to SeededLDA is the LabeledLDA model (Ramage et al., 2009).”
Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data (2014, background): “Besides the above knowledge-based topic models, Blei & McAuliffe (2007) and Ramage et al. (2009) studied topic models in a supervised setting.”
Supervised Topic Models (2007, methodology): “Researchers have used the topic-based decomposition of the collection to examine interdisciplinarity (Ramage et al., 2009), organize large collections of digital books (Mimno and McCallum, 2007), recommend purchased items (Mar-lin, 2003), and retrieve relevant documents to a query (Wei and Croft,…”