Efficient Intent Detection with Dual Sentence Encoders
By Iñigo Casanueva, Tadas Temvcinas, Daniela Gerz, Matthew Henderson, Ivan Vulic. 2020
640 citations recorded by the source corpus.
Building conversational systems in new domains and with added functionality requires resource-efficient models that work under low-data regimes (i.e., in few-shot setups). Motivated by these requirements, we introduce intent detection methods backed by pretrained dual sentence encoders such as USE and ConveRT. We demonstrate the usefulness and wide applicability of the proposed intent detectors, showing that: 1) they outperform intent detectors based on fine-tuning the full BERT-Large model or using BERT as a fixed black-box encoder on three diverse intent detection data sets; 2) the gains are especially pronounced in few-shot setups (i.e., with only 10 or 30 annotated examples per intent); 3) our intent detectors can be trained in a matter of minutes on a single CPU; and 4) they are stable across different hyperparameter settings. In hope of facilitating and democratizing research focused on intention detection, we release our code, as well as a new challenging single-domain intent detection dataset comprising 13,083 annotated examples over 77 intents.
Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Richard Socher
- Caiming Xiong
- Mohit Bansal
- Ivan Vulic
- Luke S. Zettlemoyer
- Percy Liang
- Yoshua Bengio
In Information Systems
- Caiming Xiong
- Richard Socher
- Goran Glavas
- Dilek Z. Hakkani-Tür
- Philip S. Yu
- Matthew Henderson
- Anna Korhonen
- Prakhar Gupta
In Natural Language Processing
- Richard Socher
- Caiming Xiong
- Ivan Vulic
- Jianfeng Gao
- Chien-Sheng Wu
- Dilek Z. Hakkani-Tür
- Mohit Bansal
- Maxine Eskénazi
In Dialogue & Conversational AI
- Dilek Z. Hakkani-Tür
- Philip S. Yu
- Tsung-Hsien Wen
- Matthew Henderson
- Maxine Eskénazi
- Pei-hao Su
- Caiming Xiong
- Shikib Mehri
Citations in the corpus
- ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning (2021, neutral): “– GEM (en): Wiki-Lingua (Faisal Ladhak & McKeown, 2020), WenNLG (Gardent et al., 2017; Castro Ferreira et al., 2020), CommonGEN (Lin et al., 2020), E2E (Dušek et al., 2019), DART (Radev et al., 2020), ToTTo (Parikh et al., 2020), Wiki-Auto (Jiang et al., 2020), TurkCorpus(Xu et al., 2016) – DialoGL”
- Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System (2021, positive): “IFS-I NTENT is converted from BANKING77 3 (Casanueva et al., 2020), which is a single-domain intent detection dataset comprising 13,083 annotated examples over 77 intents (average: 170 examples per intent).”
- Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, background): “As shown in Table 4 D B ) and imbalanced version ( C150 I ), BANKING77 , HWU64 (H64) , and one text classification dataset, i.e., Stackoverflow (S20) .”
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (2024, positive): “…datasets that we use are as follows: AmazonReviews (McAuley & Leskovec, 2013a), AmazonCounterfactual (O’Neill et al., 2021), Banking77 (Casanueva et al., 2020), Emotion (Saravia et al., 2018), IMDB (Maas et al., 2011), MTOPDomain/MTOPIntent (Li et al., 2021), ToxicConversations…”
- DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue (2020, background): “Casanueva et al. (2020) forego a validation set when using these datasets for intent prediction and instead only use a training and testing set.”
- Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022, methodology): “…slot filling based on public NLU benchmarks: For intent detection, we collect six datasets that carry intent annotations: HWU (Liu et al., 2019), BANKING (Casanueva et al., 2020), CLINC (Lar-son et al., 2019), SNIPS (Coucke et al., 2018), AITS (Hemphill et al., 1990), and TOP (Gupta et al., 2018).”
- Probing Pretrained Language Models for Lexical Semantics (2020, methodology): “…at the lexical level aligns well with prior observations on using BERT directly as a sentence encoder (Qiao et al., 2019; Singh et al., 2019; Casanueva et al., 2020): while [CLS] is useful for sentence-pair classification tasks, using [CLS] as a sentence representation produces inferior…”
- Long-context LLMs Struggle with Long In-context Learning (2024, neutral): “To fairly evaluate the open-source and API-based models with a series of input lengths, we sample the same example set for all the models with labels distributed evenly to ensure an Table 2: The overview of the evaluated models.”
- Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020, methodology): “…Few-shot classification (Fei-Fei et al., 2006; Vinyals et al., 2016b) has been applied to text classification tasks (Deng et al., 2019; Geng et al., 2019; Xu et al., 2019), and few-shot intent detection is also studied but without OOS (Luo et al., 2018; Xia et al., 2020; Casanueva et al., 2020).”
- Multilingual and Cross-Lingual Intent Detection from Spoken Data (2021, background): “Therefore, due to the large number of executed experiments and comparisons in this work, and preliminary results which corroborated the findings from prior work (Casanueva et al., 2020), we opt”
- ConveRT: Efficient and Accurate Conversational Representations from Transformers (2019, background): “Shopping Company FAQ (Casanueva et al., 2020), with different intent classifiers (Bunk et al., 2020), and in another dialog task: turn-based value extraction (Coope et al., 2020; Bunk et al., 2020).”
- InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022, methodology): “We calculate Model Accuracy ConvERT (Casanueva et al., 2020) 83.32 ConvERT + USE (Casanueva et al., 2020) 85.19 Example-Driven (Mehri and Eric, 2021) 85.95 PPTOD base (Su et al., 2022b) 82.81 PPTOD large (Su et al., 2022b) 84.12 DIAL-BART0 (Ours) 84.30 BART0 (zero-shot) 14.72 DIAL-BART0 (Ours,…”
- MTEB: Massive Text Embedding Benchmark (2022, background): “Banking77 (Casanueva et al., 2020) Dataset composed of online banking queries annotated with their corresponding intents.”
- RAFT: A Real-World Few-Shot Text Classification Benchmark (2021, background): “Banking77 [7] contains online banking customer service queries annotated with their intents.”
- Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs (2026, unknown): “…AmazonCounterfactualClassification [51] Classification Accuracy AmazonPolarityClassification [43] Classification Accuracy Banking77Classification [6] Classification Accuracy DBpediaClassification [23] Classification Accuracy EmotionClassification [58] Classification Accuracy ImdbClassification…”