Topic modeling: beyond bag-of-words

By Hanna M. Wallach. 2006

1,250 citations recorded by the source corpus.

Subfields: Artificial Intelligence, General Social Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

Citations in the corpus

Ask the GRU: Multi-task Learning for Deep Text Recommendations (2016, methodology): “Such methods are unable to fully leverage the text content, being limited to bag-of-words suﬃcient statistics [17], and furthermore unsupervised learning is unlikely to focus on the aspects of text relevant for content recommendation.”
Joint Deep Modeling of Users and Items Using Reviews for Recommendation (2017, background): “However, in many text modeling applications, word order is crucial [32].”
TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency (2016, background): “While bi-gram topic models have been proposed (Wallach, 2006), higher order models quickly become intractable.”
An overview of topic modeling and its current applications in bioinformatics (2016, background): “For example, a new topic model proposed by Wallach (2006) relaxes the BoW assumption and assumes that a word is generated by a topic depending on its previous word.”
Modeling online reviews with multi-grain topic models (2008, methodology): “This problem can be addressed by explicitly modeling topic transitions [5, 15, 33, 32, 28, 16], but these topic n-gram models are considerably more computationally expensive.”
Scene Classification Based on the Multifeature Fusion Probabilistic Topic Model for High Spatial Resolution Remote Sensing Imagery (2015, methodology): “Inspired by the BOW model assumption [23], the probabilistic topic model”
Social Collaborative Viewpoint Regression with Explainable Recommendations (2017, methodology): “We apply a Gibbs EM sampler [47] to conditionally approximate the posterior distribution of random variables in sCVR.”
A statistical semantic language model for source code (2013, methodology): “Inspired by topic modeling by Wallach [20], we have developed an n -gram topic model that integrates the information of both local contexts (via n -grams) and global concerns (via topics).”
Replicated Softmax: an Undirected Topic Model (2009, methodology): “The hyperparameters were optimized using stochastic E M as described by [15].”
Hidden Topic Markov Models (2007, methodology): “These are the Bigram topic model [15], the LDA collocation model and the Topical n-grams model [16].”
Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology (2018, background): “Therefore, some researchers developed variations on topic modeling that consider word order (Wallach, 2006).”
Rethinking LDA: Why Priors Matter (2009, background): “Although there is empirical evidence that topic models that use asymmetric Dirichlet priors with optimized hyperparameters, such as Pachinko allocation [10] and Wallach’s topic-based language model [18], are robust to the presence of extremely common words, these studies did not establish whether th”