Studying the History of Ideas Using Topic Models

By David Leo Wright Hall, Dan Jurafsky, Christopher D. Manning. Conference on Empirical Methods in Natural Language Processing, 2008

515 citations recorded by the source corpus.

How can the development of ideas in a scientific field be studied over time? We apply unsupervised topic modeling to the ACL Anthology to analyze historical trends in the field of Computational Linguistics from 1978 to 2006. We induce topic clusters using Latent Dirichlet Allocation, and examine the strength of each topic over time. Our methods find trends in the field including the rise of probabilistic methods starting in 1988, a steady increase in applications, and a sharp decline of research in semantics and understanding between 1978 and 2001, possibly rising again after 2001. We also introduce a model of the diversity of ideas, topic entropy, using it to show that COLING is a more diverse conference than ACL, but that both conferences as well as EMNLP are becoming broader over time. Finally, we apply Jensen-Shannon divergence of topic distributions to show that all three conferences are converging in the topics they cover.

Subfields: Artificial Intelligence, General Social Sciences

View Paper: https://www.semanticscholar.org/paper/5f3bcff2f4ce3130df160719c096bfe942b06f58

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

Citations in the corpus

The ACL anthology network corpus (2009, methodology): “This corpus has already been used in a variety of experiments (Qazvinian and Radev 2008; Hall et al., 2008; Councill et al., 2008; Qazvinian et al., 2010).”
Full-Text or Abstract? Examining Topic Coherence Scores Using Latent Dirichlet Allocation (2017, methodology): “Besides being performed on abstract data, LDA has also been applied to 12,500 full-text research articles within the field of computational linguistics [15], 2,326 articles from Neural Information Processing Systems papers (NIPS) [16], and 1,060 articles within agricultural and resource economics [1”
Syntactic Topic Models (2008, background): “2004), citation (Mimno and McCallum 2007), sentiment analysis (Blei and McAuliffe 2007; Titov and McDonald 2008), corpus exploration (Hall et al. 2008), part-of-speech labeling (Toutanova and Johnson 2008), discourse segmentation (Purver et al.”
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk (2017, unknown): “to a labeling process mediated by computation known as topic modeling. Topic modeling methods are often used to categorize a set of unlabeled documents into a finite number of coherent topic clusters [7,28,63], in turn helping researchers to navigate a large corpus (HIT documents in our case) [17,18”
Interpretation and trust: designing model-driven visualizations for text analysis (2012, methodology): “Hall et al. [25] removed 64 topics and inserted 10 hand-crafted topics in order to complete their investigation; Talley et al. [46] removed poor topics and ﬂagged questionable topics in their visualization.”
Rethinking LDA: Why Priors Matter (2009, background): “There is a significant body of work applying LDA to an wide variety of tasks including analysis of news articles [14], study of the history of scientific ideas [2, 9], topic-based search interfaces1 and navigation tools for digital libraries [12].”
Topically Driven Neural Language Model (2017, methodology): “For preprocessing we tokenise words and sentence using Stanford CoreNLP (Klein and Manning, 2003), and lowercase all words.”
Termite: visualization techniques for assessing textual topic models (2012, methodology): “[12] applied LDA to study research trends in computational linguistics across 14,000 publications.”
Measuring the Evolution of a Scientific Field through Citation Frames (2018, background): “We propose that the increased use of shared evaluations, and the statistical methodology borrowed originally from electrical engineering (Hall et al., 2008; Anderson et al., 2012) frame of a paper per year using all papers published in that year.”
Discovering Attribute Shades of Meaning with the Crowd (2015, methodology): “To measure coherency, we use a text analysis metric for topic entropy (Hall et al. 2008).”
topicmodels: An R Package for Fitting Topic Models (2011, unknown): “…used for a variety of applications, including ad-hoc information retrieval (Wei and Croft 2006), geographical information retrieval (Li, Wang, Xie, Wang, and Ma 2008) and the analysis of the development of ideas over time in the field of computational linguistics (Hall, Jurafsky, and Manning 2008).”
He Said, She Said: Gender in the ACL Anthology (2012, methodology): “Following earlier work like Hall et al. (2008), we ran LDA (Blei et al., 2003) on the ACL Anthology, producing 100 generative topics.”
Stylometric Analysis of Scientific Articles (2012, background): “Hall et al. (2008) build per-year topic models over scientiﬁc literature to track the evolution of scientiﬁc ideas.”
Machine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality (2014, methodology): “…(LDA: Blei et al. (2003)) and related methods is increasingly being used in user-focused tasks, in contexts such as the evaluation of scientiﬁc impact (McCallum et al., 2006; Hall et al., 2008), trend analysis (Bolelli et al., 2009; Lau et al., 2012a) and document search (Wang et al., 2007).”
Learning Dynamic Author Representations with Temporal Language Models (2019, background): “Besides, various works studied temporal vocabulary evolution - according to semantic graph transformations in [19] -, or thematic shifts in author communities - according to the dominant topics per time-step in [20].”