Diverse Few-Shot Text Classification with Multiple Metrics

By Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, G. Tesauro, Haoyu Wang, Bowen Zhou. 2018

225 citations recorded by the source corpus.

We study few-shot learning in natural language domains. Compared to many existing works that apply either metric-based or optimization-based meta-learning to image domain with low inter-task variance, we consider a more realistic setting, where tasks are diverse. However, it imposes tremendous difficulties to existing state-of-the-art metric-based algorithms since a single metric is insufficient to capture complex task variations in natural language domain. To alleviate the problem, we propose an adaptive metric learning approach that automatically determines the best weighted combination from a set of metrics obtained from meta-training tasks for a newly seen few-shot task. Extensive quantitative evaluations on real-world sentiment analysis and dialog intent classification datasets demonstrate that the proposed method performs favorably against state-of-the-art few shot learning algorithms in terms of predictive accuracy. We make our code and data available for further study.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2020, methodology): “Other approaches for few-shot learning in NLP include exploiting examples from related tasks (Yu et al., 2018; Gu et al., 2018; Dou et al., 2019; Qian and Yu, 2019; Yin et al., 2019) and using data augmentation (Xie et al.”
Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System (2021, neutral): “Recently, few-shot text classiﬁcation (Yu et al., 2018; Geng et al., 2019; Xia et al., 2020a) has attracted increasing attention from the NLP community since it is unlikely to have large-scale labeled data for new classes in the real world.”
Continual Few-Shot Learning for Text Classification (2021, methodology): “In the language domain, Yu et al. (2018) proposed to use a weighted combination of multiple metrics obtained from meta-training tasks for inferring on a newly-seen few-shot task.”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “Recently, Yu et al. (2018) propose a multi-metric method for few-shot text classiﬁcation.”
FLEX: Unifying Evaluation for Few-Shot NLP (2021, background): “Domain transfer has also been studied on many tasks [3, 4], including dialogue intent detection & slot tagging [31], sentiment classification [77], NLI [21], and machine translation [27, 58].”
Generalizing from a Few Examples (2019, methodology): “Example applications include parsing [64], translation [65], sentence completion (which fills in the blanks using a word chosen from a provided set) [97, 138], sentiment classification from short reviews [150, 157], user intent classification for dialog systems [157], criminal charge prediction [61]”
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (2019, methodology): “, 2018), and text classification (Yu et al., 2018) with meta-learning approaches (Schmidhuber, 1987; Finn et al.”
Trustworthy AI: From Principles to Practices (2021, background): “of few-shots learning is to generalize model to new tasks using only a few examples [78, 348, 371].”
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022, background): “Learning text classification with few labeled examples has been a problem of interest in many applications (Xu et al., 2018; Yu et al., 2018).”
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2019, methodology): “We use the Books, DVD, Electronics, and Kitchen & Housewares domains, which are commonly used domains in the literature (Yu et al., 2018).”
Bridging the domain gap in cross-lingual document classification (2019, background): “The study of weakly supervised learning in language processing is relatively new (Johnson & Zhang, 2016; Yu et al., 2018).”
Few-Shot Named Entity Recognition: A Comprehensive Study (2020, background): “Meta-learning (Ravi and Larochelle, 2017) have shown promising results for few-shot image classi-ﬁcation (Tian et al., 2020) and sentence classiﬁcation (Yu et al., 2018; Geng et al., 2019).”
Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification (2019, unknown): “Yu et al. (2018) attempt to adopt FSL for text classification and achieve promising results.”
Induction Networks for Few-Shot Text Classification (2019, methodology): “Note that for ARSC, the support set for testing is fixed by Yu et al. (2018).”
Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network (2020, methodology): “𝒙 (𝟏) 𝒚 (𝟏) 𝒙 (𝟐) 𝒚 (𝟐) 𝒙 Previous few-shot learning studies mainly focused on classiﬁcation problems, which have been widely explored with similarity-based methods (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Yan et al., 2018; Yu et al., 2018).”
A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classification (2019, background): “Recently, Few-Shot Learning on text data has been explored using metric learning (Yu et al., 2018; Jiang et al., 2018).”
Personalizing Dialogue Agents via Meta-Learning (2019, methodology): “Meta-learning for NLP application is less common, and it has been applied in semantic parsing task (Huang et al., 2018), machine translation for low resource language (Gu et al., 2018), and for text classiﬁcation (Yu et al., 2018).”