Graph-Sparse LDA: A Topic Model with Structured Sparsity

By Finale Doshi-Velez, Byron C. Wallace, Ryan P. Adams. 2014

58 citations recorded by the source corpus.

Topic modeling is a powerful tool for uncovering latent structure in many domains, including medicine, finance, and vision. The goals for the model vary depending on the application: sometimes the discovered topics are used for prediction or another downstream task. In other cases, the content of the topic may be of intrinsic scientific interest. Unfortunately, even when one uses modern sparse techniques, discovered topics are often difficult to interpret due to the high dimensionality of the underlying space. To improve topic interpretability, we introduce Graph-Sparse LDA, a hierarchical topic model that uses knowledge of relationships between words (e.g., as encoded by an ontology). In our model, topics are summarized by a few latent concept-words from the underlying graph that explain the observed words. Graph-Sparse LDA recovers sparse, interpretable summaries on two real-world biomedical datasets while matching state-of-the-art prediction performance.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, methodology): “follow-ups [58], [59], [60], this technique was also applied to visualize relationships learned between words by the word2vec language model [61].”
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge (2018, background): “In contrast to deep neural networks, an alternative is to restrict the models to belong to a family that is inherently easy to interpret (Caruana et al., 2015; Doshi-Velez et al., 2014; Letham et al., 2015).”
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2017, background): “This natural, expected property may be beneficially used to make fine-grained distinctions since relative comparisons between related concepts are a good interpretative tool (Kim et al., 2015; Doshi-Velez et al., 2015; Tibshirani, 1994; Salvatore et al., 2014).”
How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation (2018, background): “…with simple, human-simulatable functions [Caruana et al., 2015, Kim et al., 2015a, R¨uping, 2006, Bucilu et al., 2006, Ustun and Rudin, 2016, Doshi-Velez et al., 2015, Kim et al., 2015b, Krakovna and Doshi-Velez, 2016, Hughes et al., 2016, Jung et al., 2017], various kinds of logic-based…”
Predicting Complications in Critical Care Using Heterogeneous Clinical Data (2016, methodology): “Examples of such techniques include priors from ontologies in deep learning [24] and the use of hierarchical priors in topic modeling [40].”
TCAV: R ELATIVE CONCEPT IMPORTANCE TESTING WITH L INEAR C ONCEPT A CTIVATION V ECTORS (2017, background): “Similar forms of sparsity (i.e., only considering a few concepts at a time) are used in many existing interpretable models (12; 7; 28; 31; 29; 4).”
Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization (2017, background): “Elsewhere, Doshi-Velez et al. (2015) proposed a variant of LDA that exploits a priori known tree-”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey (2020, methodology): “3) Sparse LDA and Discriminant Analysis: A Bayesian non-parametric model, Graph-Sparse LDA, was introduced in [101] to find interpretable, predictive topic summaries to textual categories on datasets with hierarchical labeling.”
An Evaluation of the Human-Interpretability of Explanation (2019, methodology): “rformance. Common types of explanation include regressions with simple, human-simulatable functions [Caruana et al., 2015, Kim et al., 2015a, Ruping, 2006, Bucilu et al., 2006, Ustun and Rudin, 2016, Doshi-Velez et al., 2015, Kim et al.,¨ 2015b, Krakovna and Doshi-Velez, 2016, Hughes et al., 2016, J”
Incorporating Knowledge Graph Embeddings into Topic Modeling (2017, unknown): “Recently, (Doshi-Velez, Wallace, and Adams 2015) proposed a method for achieving interpretability by exploiting controlled structured vocabularies in which words are organized into tree-structured hierarchies.”