Out-of-Domain Detection for Natural Language Understanding in Dialog Systems
By Yinhe Zheng, Guanyi Chen, Minlie Huang. 2019
140 citations recorded by the source corpus.
Natural Language Understanding (NLU) is a vital component of dialogue systems, and its ability to detect Out-of-Domain (OOD) inputs is critical in practical applications, since the acceptance of the OOD input that is unsupported by the current system may lead to catastrophic failure. However, most existing OOD detection methods rely heavily on manually labeled OOD samples and cannot take full advantage of unlabeled data. This limits the feasibility of these models in practical applications. In this paper, we propose a novel model to generate high-quality pseudo OOD samples that are akin to IN-Domain (IND) input utterances and thereby improves the performance of OOD detection. To this end, an autoencoder is trained to map an input utterance into a latent code. Moreover, the codes of IND and OOD samples are trained to be indistinguishable by utilizing a generative adversarial network. To provide more supervision signals, an auxiliary classifier is introduced to regularize the generated OOD samples to have indistinguishable intent labels. Experiments show that these pseudo OOD samples generated by our model can be used to effectively improve OOD detection in NLU. Besides, we also demonstrate that the effectiveness of these pseudo OOD data can be further improved by efficiently utilizing unlabeled data.
Subfields: Artificial Intelligence, Natural Language Processing, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dustin Tran
- Balaji Lakshminarayanan
- Shreyas Padhy
- Tania Bedrax-Weiss
- Zi Lin
- Jeremiah Zhe Liu
- Nikolaos D. Tselikas
- Konstantinos I. Roumeliotis
In Natural Language Processing
- Nikolaos D. Tselikas
- Konstantinos I. Roumeliotis
- Tania Bedrax-Weiss
- Balaji Lakshminarayanan
- Dustin Tran
Citations in the corpus
- Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “Detecting Out-of-Scope Intent in Conversational Language Understanding To validate the method beyond image modalities, we also evaluate SNGP on a practical language understanding task where uncertainty quantification is of natural importance: dialog intent detection [44, 78, 82, 84].”
- ChatGPT and Open-AI Models: A Preliminary Review (2023, background): “Inability to recognize or understand images: While ChatGPT is capable of processing text, it does not have the ability to recognize or understand images, which can limit its usefulness in certain applications [88].”