Fast Domain Adaptation for Goal-Oriented Dialogue Using a Hybrid Generative-Retrieval Transformer
By Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz. 2020
18 citations recorded by the source corpus.
Goal-oriented dialogue systems are now widely adopted in industry, where practical aspects of using them becomes of key importance. As such, it is expected from such systems to fit into a rapid prototyping cycle for new products and domains. For data-driven dialogue systems (especially those based on deep learning) that amounts to maintaining production-level performance having been provided with a few ‘seed’ dialogue examples, normally referred to as data efficiency.With extremely data-dependent deep learning methods, the most promising way to achieve practical data efficiency is transfer learning—i.e., leveraging a greater, highly represented data source for training a base model, then fine-tuning it to available in-domain data.In this paper, we present a hybrid generative-retrieval model that can be trained using transfer learning. By using GPT-2 as the base model and fine-tuning it to the multidomain MetaLWOz dataset, we obtain a robust dialogue model able to perform both response generation and ranking 1. Combining both, it outperforms several competitive generative-only and retrieval-only baselines, measured by language modeling quality on MetaLWOz as well as in goal- oriented metrics (Intent/Slot Fl-scores) on the MultiWoz corpus.
Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Dialogue & Conversational AI
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hannaneh Hajishirzi
- Chitta Baral
- Yejin Choi
- Swaroop Mishra
- Daniel Khashabi
- Noah A. Smith
- Luke S. Zettlemoyer
- Mohit Bansal
In Representation & Self-Supervised Learning
- Hannaneh Hajishirzi
- Noah A. Smith
- Yejin Choi
- Yizhong Wang
- Daniel Khashabi
- Arjun Ashok
- Chitta Baral
- Shailaja Keyur Sampat
In Dialogue & Conversational AI
Citations in the corpus
- Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022, methodology): “Table 9: An example task in the Dialogue Act Recognition category of our dataset, adopted from MetaLWOz (Shalyminov et al., 2020).”
- CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking (2026, unknown): “(Eric et al., 2017) 2,424 302 MetaLWOZ (Shalyminov et al., 2020) 10,000 MS-DC (Li et al., 2018) 9,985 MuDoCo (Martin et al., 2020) 6,032 690 MultiDoGO (Peskov et al., 2019) 10,000 1,150 MultiWOZ 2.1 (Eric et al., 2020) 8,434 999 MultiWOZ 2.2 (Zang et al., 2020) 8,437 1,000 SGD (Rastogi et al., 2020)”