Train Once, Test Anywhere: Zero-Shot Learning for Text Classification

By Pushpankar Kumar Pushp, Muktabh Mayank Srivastava. 2017

103 citations recorded by the source corpus.

Zero-shot Learners are models capable of predicting unseen classes. In this work, we propose a Zero-shot Learning approach for text categorization. Our method involves training model on a large corpus of sentences to learn the relationship between a sentence and embedding of sentence's tags. Learning such relationship makes the model generalize to unseen sentences, tags, and even new datasets provided they can be put into same embedding space. The model learns to predict whether a given sentence is related to a tag or not; unlike other classifiers that learn to classify the sentence as one of the possible classes. We propose three different neural networks for the task and report their accuracy on the test set of the dataset used for training them as well as two other standard datasets for which no retraining was done. We show that our models generalize well across new unseen classes in both cases. Although the models do not achieve the accuracy level of the state of the art supervised models, yet it evidently is a step forward towards general intelligence in natural language processing.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Lichao Sun
3. Ruiqi Zhong
4. Zheng Zhang
5. Daniel Klein
6. Qian Li
7. Luke S. Zettlemoyer
8. Kristy Lee

In Computer Vision and Pattern Recognition

1. Philip S. Yu
2. Zheng Zhang
3. Daniel Klein
4. Dacheng Tao
5. Jianxin Li
6. Congyin Xia
7. Renyu Yang
8. Kristy Lee

In Natural Language Processing

1. Philip S. Yu
2. Ruiqi Zhong
3. Lichao Sun
4. Zheng Zhang
5. Qian Li
6. Jianxin Li
7. Congyin Xia
8. Renyu Yang

In Representation & Self-Supervised Learning

1. Philip S. Yu
2. Jianxin Li
3. Congyin Xia
4. Renyu Yang
5. Zhiquan Ye
6. Qian Li
7. Zheng Zhang
8. Daniel Klein

Citations in the corpus

• Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, background): “Nevertheless, our procedure is not bullet-proof and one can argue that our notion of unseen tasks, though harsher than prior works (Yin et al., 2019; Pushp and Srivastava, 2017), is still lenient.”
• A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “The main idea is to infer the features through learning kinds of semantic knowledge, such as learning relationships among classes [288] and incorporating class descriptions [170].”
• Zero-shot Text Classification via Reinforced Self-training (2020, background): “of the text, and takes the max similarity as final matching score; (3) FC and RNN+FC refers to the architecture 1 and architecture 2 proposed in (Pushp and Srivastava, 2017).”