Weakly-Supervised Neural Text Classification

By Yu Meng, Jiaming Shen, Chao Zhang, Jiawei Han. 2018

200 citations recorded by the source corpus.

Deep neural networks are gaining increasing popularity for the classic text classification task, due to their strong expressive power and less requirement for feature engineering. Despite such attractiveness, neural text classification models suffer from the lack of training data in many real-world applications. Although many semi-supervised and weakly-supervised text classification models exist, they cannot be easily applied to deep neural models and meanwhile support limited supervision types. In this paper, we propose a weakly-supervised method that addresses the lack of training data in neural text classification. Our method consists of two modules: (1) a pseudo-document generator that leverages seed information to generate pseudo-labeled documents for model pre-training, and (2) a self-training module that bootstraps on real unlabeled data for model refinement. Our method has the flexibility to handle different types of weak supervision and can be easily integrated into existing deep neural models for text classification. We have performed extensive experiments on three real-world datasets from different domains. The results demonstrate that our proposed method achieves inspiring performance without requiring excessive training data and outperforms baseline methods significantly.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiawei Han
2. Erik Cambria
3. Jianfeng Gao
4. Diyi Yang
5. Yu Meng
6. Shervin Minaee
7. Chao Zhang
8. Jiaao Chen

In Natural Language Processing

1. Jiawei Han
2. Erik Cambria
3. Diyi Yang
4. Jianfeng Gao
5. Yu Meng
6. Shervin Minaee
7. Jiaao Chen
8. Chao Zhang

Citations in the corpus

• Generating Training Data with Language Models: Towards Zero-Shot Language Understanding (2022, methodology): “The idea of using generated texts as training data has been explored in topic classification with bag-of-words or LSTMbased language models (Meng et al., 2018; 2019) which do not have enough capacity to generate quality training data for challenging NLU tasks.”
• MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification (2020, background): “…al., 2019); (2) encouraging models to output confident predictions on unlabeled data for self-training like (Lee, 2013; Grandvalet and Bengio, 2004; Meng et al., 2018); (3) performing consistency training after adding adversarial noise (Miy-ato et al., 2019, 2017) or data augmentations (Xie et…”
• Weakly-Supervised Hierarchical Text Classification (2018, methodology): “• Bag-of-words + reordering: We first generate bag-of-words pseudo documents as in the previous method, and then use the globally trained LSTM language model to reorder the pseudo documents by greedily putting the word with the highest probability at the end of the current sequence.”
• A Survey on Programmatic Weak Supervision (2022, background): “For example, in text applications, users write keyword-or regex-based LFs that assign corresponding labels to the data points that contains the keyword or matches the specified regular expression [ Ratner et al. , 2017; Meng et al. , 2018; Awasthi et al. , 2020 ] .”
• Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification (2019, background): “Short text classification can be widely applied in many domains, ranging from sentiment analysis to news tagging/categorization and query intent classification (Aggarwal and Zhai, 2012; Meng et al., 2018).”
• Deep Learning--based Text Classification (2020, methodology): “In general, data augmentation using VAE or other models [164, 165] is widely used for semi-supervised or weakly supervised text classification.”
• Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning (2022, background): “More specifically related to the idea of generating training data with language models, early meth-ods in text classification use bag-of-words or LSTM-based language models (Meng et al., 2018; 2019) to generate class-conditioned texts as training data.”
• Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Keywords inspired self-training methods like WeSTClass [165], LOTClass [168], and X-Class [255].”