Zero-shot Cross-lingual Dialogue Systems with Transferable Latent Variables

By Zihan Liu, Jamin Shin, Yan Xu, Genta Indra Winata, Peng Xu, Andrea Madotto, Pascale Fung. 2019

71 citations recorded by the source corpus.

Despite the surging demands for multilingual task-oriented dialog systems (e.g., Alexa, Google Home), there has been less research done in multilingual or cross-lingual scenarios. Hence, we propose a zero-shot adaptation of task-oriented dialogue system to low-resource languages. To tackle this challenge, we first use a set of very few parallel word pairs to refine the aligned cross-lingual word-level representations. We then employ a latent variable model to cope with the variance of similar sentences across different languages, which is induced by imperfect cross-lingual alignments and inherent differences in languages. Finally, the experimental results show that even though we utilize much less external resources, our model achieves better adaptation performance for natural language understanding task (i.e., the intent detection and slot filling) compared to the current state-of-the-art model in the zero-shot scenario.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Measuring Association Between Labels and Free-Text Rationales (2020, methodology): “Method Following related work (Wang et al., 2019; Lakshmi Narayan et al., 2019; Liu et al., 2019b), we add zero-mean Gaussian noise N (0 , σ 2 ) to each input embedding in the I → OR encoder at inference time.”
Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning (2020, methodology): “Recently, cross-lingual methods have been applied to multiple NLP tasks, such as task-oriented dialogue systems (Liu et al., 2019a,b), part-of-speech tagging (Wisniewski et al., 2014; Zhang et al., 2016; Kim et al., 2017), named entity recognition (Ni et al., 2017; Xie et al., 2018), abstractive…”
Efficient Intent Detection with Dual Sentence Encoders (2020, background): “Transfer learning on top of pretrained sentence encoders (Devlin et al., 2019; Liu et al., 2019b, inter alia ) has now established as the mainstay paradigm aiming to mitigate the bottleneck with scarce in-domain data.”
Five sources of bias in natural language processing (2021, methodology): “These methods include, for example, multi-lingual representations (Devlin et al., 2019; Nozza et al., 2020) and the zero-shot learning they enable (e.g., Bianchi et al., 2021; Jebbara & Cimiano, 2019; Liu et al., 2019, inter alia).”
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization (2021, background): “Recently, conducting a second pre-training step on large-scale language models (e.g., BERT (De-vlin et al., 2019), RoBERTa (Liu et al., 2019a)) has proven to be effective for domain adaptation tasks (Lee et al., 2020; Gururangan et al., 2020).”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, background): “…2017; Shah et al., 2019; Wu et al., 2019a; Liu et al., 2020) and target languages (Mrkˇsi´c et al., 2017; Schuster et al., 2019; Chen et al., 2018; Liu et al., 2019b), and large pre-trained language models are shown to possess the capability to quickly adapt to task-oriented dialogue tasks by…”