Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge

By Ryan J. Gallagher, Kyle Reing, David C. Kale, Greg Ver Steeg. 2016

191 citations recorded by the source corpus.

While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters. Such model complexity issues only compound when trying to generalize generative models to incorporate human input. We introduce Correlation Explanation (CorEx), an alternative approach to topic modeling that does not assume an underlying generative model, and instead learns maximally informative topics through an information-theoretic framework. This framework naturally generalizes to hierarchical and semi-supervised extensions with no additional modeling assumptions. In particular, word-level domain knowledge can be flexibly incorporated within CorEx through anchor words, allowing topic separability and representation to be promoted with minimal human intervention. Across a variety of datasets, metrics, and experiments, we demonstrate that CorEx produces topics that are comparable in quality to those produced by unsupervised and semi-supervised variants of LDA.

Subfields: Artificial Intelligence, General Social Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Aram Galstyan
2. Greg Ver Steeg
3. Shuyang Gao
4. Rob Brekelmans
5. Daniel Moyer
6. Roman Egger
7. Joanne Yu
8. Fei Ye

In General Social Sciences

1. Aram Galstyan
2. Rob Brekelmans
3. Shuyang Gao
4. Roman Egger
5. Joanne Yu
6. Greg Ver Steeg

Citations in the corpus

• Invariant Representations without Adversarial Training (2018, methodology): “An unsupervised method, CorEx, also has a similar extension (Anchored Corex [9]), in which latent factors can be driven toward specific targets.”
• Auto-Encoding Total Correlation Explanation (2018, methodology): “Our work is based on the unsupervised principle of Correlation Explanation, but other variants of CorEx also exist, such as Anchored CorEx (Gallagher et al., 2017), in this case, the anchoring worked on a different term and were used in semi-supervised settings.”
• Evading the Adversary in Invariant Representation (2018, methodology): “An unsupervised method, CorEx, also has a similar extension (Anchored Corex [9]), in which latent factors can be driven toward specific targets.”
• A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts (2022, methodology): “Despite the robustness of topic modeling algorithms, existing literature relies primarily on one single model, with LDA being the dominant method (Gallagher et al., 2017) and is typically viewed as the standard approach.”