CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection

By Congying Xia, Chenwei Zhang, Hoang Nguyen, Jiawei Zhang, Philip S. Yu. 2020

48 citations recorded by the source corpus.

In this paper, we formulate a more realistic and difficult problem setup for the intent detection task in natural language understanding, namely Generalized Few-Shot Intent Detection (GFSID). GFSID aims to discriminate a joint label space consisting of both existing intents which have enough labeled data and novel intents which only have a few examples for each class. To approach this problem, we propose a novel model, Conditional Text Generation with BERT (CG-BERT). CG-BERT effectively leverages a large pre-trained language model to generate text conditioned on the intent label. By modeling the utterance distribution with variational inference, CG-BERT can generate diverse utterances for the novel intents even with only a few utterances available. Experimental results show that CG-BERT achieves state-of-the-art performance on the GFSID task with 1-shot and 5-shot settings on two real-world datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Caiming Xiong
2. Richard Socher
3. Philip S. Yu
4. Wenpeng Yin
5. Chien-Sheng Wu
6. Mohit Bansal
7. Kazuma Hashimoto
8. Jianfeng Gao

In Computer Vision and Pattern Recognition

1. Caiming Xiong
2. Philip S. Yu
3. Richard Socher
4. Dacheng Tao
5. Wenpeng Yin
6. Kazuma Hashimoto
7. Luc Van Gool
8. Trevor Darrell

Citations in the corpus

• Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System (2021, neutral): “All previous few-shot learning models (Snell et al., 2017; Gidaris and Komodakis, 2018; Yin et al., 2020; Xia et al., 2020b; Nguyen et al., 2020) fail to solve this problem since they relied on the large-scale labeled data of base classes to train a robust system.”
• Meta-learning for Few-shot Natural Language Processing: A Survey (2020, background): “Both (Xia et al., 2018, 2020) used two intents as few-shot classes and other intents for training.”
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020, methodology): “…Few-shot classification (Fei-Fei et al., 2006; Vinyals et al., 2016b) has been applied to text classification tasks (Deng et al., 2019; Geng et al., 2019; Xu et al., 2019), and few-shot intent detection is also studied but without OOS (Luo et al., 2018; Xia et al., 2020; Casanueva et al., 2020).”
• Composed Variational Natural Language Generation for Few-shot Intents (2020, methodology): “Xia et al. (2020) provide the first work that combines CVAE with BERT to generate utterances for generalized few-shot intent detection.”
• Semantic Matching and Aggregation Network for Few-shot Intent Detection (2020, background): “This problem could further lead to overfitting either to seen intents or novel intents, especially in the challenging Generalized Few-shot Intent Detection (GFSID) setting (Xia et al., 2020) where both seen and novel intents are existent in a joint label space during inference.”