Statistical topic models for multi-label document classification

By Timothy N. Rubin, America Chambers, Padhraic Smyth, Mark Steyvers. Machine-mediated learning, 2011

354 citations recorded by the source corpus.

Machine learning approaches to multi-label document classification have to date largely relied on discriminative modeling techniques such as support vector machines. A drawback of these approaches is that performance rapidly drops off as the total number of labels and the number of labels per document increase. This problem is amplified when the label frequencies exhibit the type of highly skewed distributions that are often observed in real-world datasets. In this paper we investigate a class of generative statistical topic models for multi-label documents that associate individual word tokens with different labels. We investigate the advantages of this approach relative to discriminative models, particularly with respect to classification problems involving large numbers of relatively rare labels. We compare the performance of generative and discriminative approaches on document labeling tasks ranging from datasets with several thousand labels to datasets with tens of labels. The experimental results indicate that probabilistic generative models can achieve competitive multi-label classification performance compared to discriminative methods, and have advantages for datasets with many labels and skewed label frequencies.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/aac214fd3bef79f4f4bd50eaefe3f679097c5657

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhi-Hua Zhou
2. Min-Ling Zhang
3. Ayan Acharya
4. Wenfu Dong
5. L. Tang
6. Raymond J. Mooney
7. Shaowen Yao
8. Lin Liu

In Signal Processing

1. Zhi-Hua Zhou
2. Wenfu Dong
3. L. Tang
4. Raymond J. Mooney
5. Shaowen Yao
6. Lin Liu
7. Min-Ling Zhang
8. Ayan Acharya

Citations in the corpus

• Lift: Multi-Label Learning with Label-Specific Features (2011, background): “…applications, such as text categorization where a news document could cover several topics including politics, economics, and reform [31], [39], [40], multimedia content annotation where one image could demonstrate several scenes including beach and building [3], [4], [49],…”
• A Review on Multi-Label Learning Algorithms (2014, background): “html (Machine Learning Journal Special Issue on Learning from Multi-Label Data [17], [37], [65], [68], [120], [126], [178], [199]) http://mulan.”
• An overview of topic modeling and its current applications in bioinformatics (2016, result): “Moreover, compared with other classification approaches such as support vector machine (SVM) (Rubin et al. 2011), the classification result of a topic model under certain conditions shows competitive performance.”
• Active Multitask Learning Using Both Latent and Supervised Shared Topics (2014, methodology): “The unsupervised LDA has been extended to account for supervision by labeling each document with its set of topics [31, 34].”