Syntactic Topic Models

By Jordan L. Boyd-Graber, David M. Blei. 2008

218 citations recorded by the source corpus.

We develop the syntactic topic model (STM), a nonparametric Bayesian model of parsed documents. The STM generates words that are both thematically and syntactically constrained, which combines the semantic insights of topic models with the syntactic information available from parse trees. Each word of a sentence is generated by a distribution that combines document-specific topic weights and parse-tree-specific syntactic transitions. Words are assumed to be generated in an order that respects the parse tree. We derive an approximate posterior inference method based on variational methods for hierarchical Dirichlet processes, and we report qualitative and quantitative results on both synthetic data and hand-parsed documents.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (2018, background): “, determiners require less context than nouns (Boyd-Graber and Blei, 2009).”
BTM: Topic Modeling over Short Texts (2014, background): “Index Terms—Short text, topic model, biterm, online algorithm, content analysis Ç”
Text: now in 2D! A framework for lexical expansion with contextual similarity (2013, methodology): “This includes computational models for topicality (Deerwester et al., 1990; Hofmann, 1999; Blei et al., 2003), and language models that incorporate topical (as well as syntactic) information (Boyd-Graber and Blei, 2008; Tan et al., 2012).”
Automatic Caption Generation for News Images (2013, background): “a distribution that combines document-specific topics and parse-tree-specific syntactic transitions (Boyd-Graber and Blei, 2009).”
Interactive Topic Modeling (2011, background): “When presented with poor topics learned from data, users can offer a number of complaints:2 these documents should have similar topics but don’t (Daumé III, 2009); this topic should have syntactic coherence (Gruber et al., 2007; Boyd-Graber and Blei, 2008); this topic doesn’t make any sense at all (”
Topic modelling for qualitative studies (2017, background): “Syntactic topic models introduce syntactic constraints inside a document that replace the bag-of-words model with syntactic parse trees [16].”
Covariance in Unsupervised Learning of Probabilistic Grammars (2010, methodology): “Variational inference algorithms have been successfully applied to various grammar and syntax learning tasks (Kurihara and Sato, 2006; Liang et al., 2007; Headden et al., 2009; Boyd-Graber and Blei, 2010; Cohen et al., 2010,inter alia).”
Online Variational Inference for the Hierarchical Dirichlet Process (2011, methodology): “The challenge we face is that the existing coordinate ascent variational Bayes algorithms for the HDP require complicated approximation methods or numerical optimization [3, 4, 13].”
Mining Text Data (2012, methodology): “as representation of the input has led to high performance in selecting important content for multi-document summarization of news [15, 38].”
Topic Models for Image Annotation and Text Illustration (2010, background): “, by assuming that each word is generated by a distribution that combines document-specific topics and parse-treespecific syntactic transitions (Boyd-Graber and Blei, 2009).”
Tailoring Continuous Word Representations for Dependency Parsing (2014, background): “, via kmeans or the Brown et al. (1992) algorithm.”
Visual Information in Semantic Representation (2010, background): “Analogously, we could represent documents by their syntactic structure (Boyd-Graber and Blei, 2009).”
Top2Vec: Distributed Representations of Topics (2020, background): “This is one of the greatest weakness of these models, as the number of topics t or the way to estimate it is rarely known, especially for very large or unfamiliar datasets [3, 4].”
A biterm topic model for short texts (2013, background): “In essence, the conventional topic models reveal the latent topics within the text corpus by implicitly capturing the document-level word co-occurrence patterns [5, 30].”
Composition in Distributional Models of Semantics (2010, background): “Boyd-Graber and Blei (2008) integrate semantic and syntactic structure in a single model based on LDA.”