Reconstructing Capsule Networks for Zero-shot Intent Classification

By 刘晗, Xiaotong Zhang, Lu Fan, Xuandi Fu, Qimai Li, Xiao-Ming Wu, Albert Y. S. Lam. 2019

49 citations recorded by the source corpus.

Intent classification is an important building block of dialogue systems. With the burgeoning of conversational AI, existing systems are not capable of handling numerous fast-emerging intents, which motivates zero-shot intent classification. Nevertheless, research on this problem is still in the incipient stage and few methods are available. A recently proposed zero-shot intent classification method, IntentCapsNet, has been shown to achieve state-of-the-art performance. However, it has two unaddressed limitations: (1) it cannot deal with polysemy when extracting semantic capsules; (2) it hardly recognizes the utterances of unseen intents in the generalized zero-shot intent classification setting. To overcome these limitations, we propose to reconstruct capsule networks for zero-shot intent classification. First, we introduce a dimensional attention mechanism to fight against polysemy. Second, we reconstruct the transformation matrices for unseen intents by utilizing abundant latent information of the labeled utterances, which significantly improves the model generalization ability. Experimental results on two task-oriented dialogue datasets in different languages show that our proposed method outperforms IntentCapsNet and other strong baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Luke S. Zettlemoyer
2. Quoc V. Le
3. Mohit Bansal
4. Percy Liang
5. Graham Neubig
6. Hannaneh Hajishirzi
7. Jianfeng Gao
8. Jason Wei

In Computer Vision and Pattern Recognition

1. Quoc V. Le
2. Lei Li
3. Luc Van Gool
4. Luke S. Zettlemoyer
5. Jianfeng Gao
6. Dacheng Tao
7. Adams Wei Yu
8. Mohit Bansal

In Natural Language Processing

1. Andrew M. Dai
2. Lei Li
3. Quoc V. Le
4. Kelvin Guu
5. Adams Wei Yu
6. Jason Wei
7. Mohit Bansal
8. Nan Du

Citations in the corpus

• Finetuned Language Models Are Zero-Shot Learners (2021, background): “…is an established area within deep learning (Collobert et al., 2011; Luong et al., 2016; Ruder, 2017; Velay & Daniel, 2018; Clark et al., 2019b; Liu et al., 2019b, inter alia)—see Worsham & Kalita (2020) for a recent survey on MTL for NLP. Differing from prior MTL work which focuses on…”
• Pre-trained Language Models Can be Fully Zero-Shot Learners (2022, methodology): “Transferring knowledge from seen classes to unseen ones requires accurate and discriminative descriptions of all classes (Liu et al., 2019a; Xia et al., 2018) or joint embeddings of categories and documents (Nam et al., 2016).”
• Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification (2020, methodology): “Current generalized zero-shot intent classification methods (Chen et al., 2016; Kumar et al., 2017; Xia et al., 2018; Liu et al., 2019) attempt to classify test instances directly by making predictions in the pool of all the seen and unseen intents.”