Induction Networks for Few-Shot Text Classification

By Ruiying Geng, Binhua Li, Yongbin Li, Xiao-Dan Zhu, Ping Jian. 2019

196 citations recorded by the source corpus.

Text classification tends to struggle when data is deficient or when it needs to adapt to unseen classes. In such challenging scenarios, recent studies have used meta-learning to simulate the few-shot task, in which new queries are compared to a small support set at the sample-wise level. However, this sample-wise comparison may be severely disturbed by the various expressions in the same class. Therefore, we should be able to learn a general representation of each class in the support set and then compare it to new queries. In this paper, we propose a novel Induction Network to learn such a generalized class-wise representation, by innovatively leveraging the dynamic routing algorithm in meta-learning. In this way, we find the model is able to induce and generalize better. We evaluate the proposed model on a well-studied sentiment classification dataset (English) and a real-world dialogue intent classification dataset (Chinese). Experiment results show that on both datasets, the proposed model significantly outperforms the existing state-of-the-art approaches, proving the effectiveness of class-wise generalization in few-shot text classification.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System (2021, neutral): “Recently, few-shot text classiﬁcation (Yu et al., 2018; Geng et al., 2019; Xia et al., 2020a) has attracted increasing attention from the NLP community since it is unlikely to have large-scale labeled data for new classes in the real world.”
A Survey on Text Classification: From Traditional to Deep Learning (2020, methodology): “Furthermore, latent features generation [289], meta-Learning [106, 290, 291], and dynamic memory mechanism [292] are also efficient methods.”
CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection (2020, methodology): “Recent deep learning-based FSL approaches mainly fall into two categories: (1) metric-based approaches, such as Matching Networks (Vinyals et al., 2016) and Prototypical Networks (Snell et al., 2017), which tries to learn one generalizable metric to separate the classes based on the distance for all”
Few-Shot Named Entity Recognition: A Comprehensive Study (2020, methodology): “Meta-learning (Ravi and Larochelle, 2017) have shown promising results for few-shot image classi-ﬁcation (Tian et al., 2020) and sentence classiﬁcation (Yu et al., 2018; Geng et al., 2019).”
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2019, methodology): “Moreover, their application in NLP has followed a similar trend (Han et al., 2018; Yu et al., 2018; Guo et al., 2018; Mi et al., 2019; Geng et al., 2019).”
DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference (2021, background): “, 2020), domains (Bao et al., 2020; Yu et al., 2018; Geng et al., 2019), and language pairs (Gu et al.”
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020, methodology): “Meta-learning (Finn et al., 2017; Geng et al., 2019) is widely studied in the ﬁrst scenario.”
Meta-learning for Few-shot Natural Language Processing: A Survey (2020, background): “Yu et al. (2018) and Geng et al. (2019) studied two text classiﬁcation problems, each from multiple domains: one is multi-domain sentiment classiﬁcation, the other is multi-domain intent classiﬁcation.”
Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network (2020, background): “Few-shot learning in natural language process-5 ing has been explored for classiﬁcation tasks, including text classiﬁcation (Sun et al., 2019; Geng et al., 2019; Yan et al., 2018; Yu et al., 2018), entity relation classiﬁcation (Lv et al., 2019; Gao et al., 2019; Ye and Ling, 2019), and dialog act…”