Sparse Additive Generative Models of Text

By Jacob Eisenstein, Amr Ahmed, Eric P. Xing. 2011

387 citations recorded by the source corpus.

Subfields: Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Generative Models

Citations in the corpus

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models (2020, methodology): “Vocabulary Shifting (VOCAB-SHIFT) Inspired by Eisenstein et al. (2011) and Ghosh et al. (2017), we learn a 2-dimensional representation of toxicity and non-toxicity for every token in GPT-2’s vocabulary, which we then use to boost the likelihood of non-toxic tokens.”
Demographic Dialectal Variation in Social Media: A Case Study of African-American English (2016, methodology): “This model has broadly similar goals as non-latent, log-linear generative models of text that condition on document-level covariates (Monroe et al., 2008; Eisenstein et al., 2011a; Taddy, 2013).”
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification (2014, background): “, [12, 32, 13, 23, 15]); and prototype- or case-based classifiers (e.”
Graph-Sparse LDA: A Topic Model with Structured Sparsity (2014, background): “Sparse topic models [8, 9, 10] offer a partial solution to this problem by e nforcing the constraint that many of the word probabilities for a given topic should be zero.”
That’s So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets (2015, methodology): “SAGE clearly does a good job identifying annoying specific behaviors in each category.”
Dynamic Language Models for Streaming Text (2014, background): “Intuitively, this is equivalent to encouraging the deviation vector to be sparse (Eisenstein et al., 2011). shifts.”
A Model of Text for Experimentation in the Social Sciences (2016, methodology): “Our approach to this task builds on two prior efforts to incorporate general covariate information into topicmodels, the Dirichlet-multinomial regression topic model of Mimno and McCallum (2008) and the sparse additive generative model of Eisenstein, Ahmed, and Xing (2011).”
Quantifying herding effects in crowd wisdom (2014, methodology): “The exponential additive generative mechanism has been applied in modeling latent topics for text [6].”
Structural Topic Models for Open‐Ended Survey Responses (2014, methodology): “For this, we turn to the exclusivity of topic words, drawing on previous work on exclusivity and diversity in topic models (Bischof and Airoldi 2012; Eisenstein, Ahmed, and Xing 2011; Zou and Adams 2012).”
Discovering geographical topics in the twitter stream (2012, methodology): “# of latent regions [[7]] [[16]] [[6]] Topics Topics + Region Full Model 10 494 479 501 540.”
A Bayesian Mixed Effects Model of Literary Character (2014, background): “The latter have been applied speciﬁcally to the problem of estimating word probabilities with sparse additive generative (SAGE) models (Eisenstein et al., 2011), where sparse extra-linguistic effects can inﬂuence a word probability in a larger generative setting.”
Is the Left-Right Scale a Valid Measure of Ideology? (2017, methodology): “Given the short length of the text in the responses, we estimate a Sparse Additive Generative (SAGE) model (Eisenstein et al, 2011), which has been shown to yield more semantically coherent topics than LDA 10 In the first sample stage municipalities (Gemeinden) in Western Germany and municipalities…”
Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human Solutions (2023, methodology): “Table 6 presents the SAGE analysis results that distinguished the salient keywords in two datasets.”
Improving Topic Models with Latent Feature Word Representations (2015, methodology): “Applying MAP estimation to learn log-linear models for topic models is also used in SAGE (Eisenstein et al., 2011) and SPRITE (Paul and Dredze, 2015).”
Aggregating Continuous Word Embeddings for Information Retrieval (2013, background): “As a result, it has been extended or reﬁned in a count-less studies (Grifﬁths et al., 2004; Eisenstein et al., 2011; Hoffman et al., 2010).”
Interactive and interpretable machine learning models for human machine collaboration (2015, background): “Active learning [37, 128, 102], semi-supervised learning [36, 15], interactive machine learning [18, 89, 7] and a set of application-specific systems [119, 125, 42, 131, 5, 14, 7] are some examples of work that enable communication from humans to machines while improving the way to represent machine”
Mind the Gap: A Generative Approach to Interpretable Feature Selection and Extraction (2015, background): “For example, [30] builds a topic model in which each topic is described by the difference from some baseline distribution.”
Confounds and Consequences in Geotagged Twitter Data (2015, methodology): “For a more detailed view, we apply SAGE to identify the most salient lexical items for each MSA, subgrouped by age and gender.”