Data Augmentation for Spoken Language Understanding via Joint Variational Generation

By Kang Min Yoo, Youhyun Shin, Sang-goo Lee. 2018

94 citations recorded by the source corpus.

Data scarcity is one of the main obstacles of domain adaptation in spoken language understanding (SLU) due to the high cost of creating manually tagged SLU datasets. Recent works in neural text generative models, particularly latent variable models such as variational autoencoder (VAE), have shown promising results in regards to generating plausible and natural sentences. In this paper, we propose a novel generative architecture which leverages the generative power of latent variable models to jointly synthesize fully annotated utterances. Our experiments show that existing SLU models trained on the additional synthetic examples achieve performance gains. Our approach not only helps alleviate the data scarcity issue in the SLU task for many datasets but also indiscriminately improves language understanding performances for various SLU models, supported by extensive experiments and rigorous statistical testing.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Survey of Data Augmentation Approaches for NLP (2021, background): “Hou et al. (2018), Kim et al. (2019), Zhao et al. (2019), and Yoo et al. (2019) investigate DA methods for dialogue and spoken language understanding (SLU), including generative latent variable models.”
Out-of-Domain Detection for Natural Language Understanding in Dialog Systems (2019, background): “2983593 network-based NLU models are proposed, and some of these models have been applied in real-world applications [1]–[3].”
An Analysis of Simple Data Augmentation for Named Entity Recognition (2020, background): “…(Wei and Zou, 2019), or word position swap (S¸ahin and Steedman, 2018; Min et al., 2020); or create entirely artiﬁcial instances via generative models, such as variational auto encoders (Yoo et al., 2019; Mesbah et al., 2019) or back-translation models (Yu et al., 2018; Iyyer et al., 2018).”
Data Augmentation by Data Noising for Open-vocabulary Slots in Spoken Language Understanding (2019, methodology): “It is used in SLU studies (Goo et al., 2018; Yoo et al., 2018).”
Simulated Chats for Task-oriented Dialog: Learning to Generate Conversations from Instructions (2020, methodology): “Such methods include, generating paraphrased utterances of existing conversations using sequence-to-sequence generative approaches (Hou et al. 2018; Anaby-Tavor et al. 2020), generating annotations including intent-slots and dialog acts (Yoo, Shin, and Lee 2019; Yoo et al. 2020a) etc.”
Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network (2020, methodology): “For data augmentation of slot tagging, sentence generation based methods are explored to create additional labeled samples (Hou et al., 2018; Shin et al., 2019; Yoo et al., 2019).”
Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding (2021, methodology): “Here, we adopt the Gaussian distribution N (u, σ(2)) as in many existing works [40].”
Paraphrase Augmented Task-Oriented Dialog Generation (2020, background): “Such an idea led to recent augmentation work on the language understanding task (Hou et al., 2018; Kim et al., 2019; Yoo et al., 2019; Zhao et al., 2019).”
Dialog State Tracking with Reinforced Data Augmentation (2019, methodology): “There is prior work on data augmentation in NLP (Zhang et al., 2015; Kang et al., 2018; Kobayashi, 2018; Hou et al., 2018; Ray et al., 2018; Yoo et al., 2018).”
A Closer Look At Feature Space Data Augmentation For Few-Shot Intent Classification (2019, background): “For text classification, several text generation based data augmentation techniques have also been explored (Hou et al., 2018; Zhao et al., 2019; Guu et al., 2018; Yoo et al., 2018; Cho et al., 2019a).”
Data Augmentation with Atomic Templates for Spoken Language Understanding (2019, methodology): “Yoo et al. (2018) use a variational autoencoder to generate labelled language.”
Data Augmentation Approaches in Natural Language Processing: A Survey (2021, methodology): “[80] propose a novel VAE model to output the semantic slot sequence and the intent label given an utterance.”
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation (2021, background): “Some other researchers have taken the direction of perturbing the latent spaces, optionally by introducing variational inference in the architecture (Xia et al., 2020b,a; Hou et al., 2018; Yoo et al., 2019).”