Diversifying Dialogue Generation with Non-Conversational Text

By Hui Su, Xiaoyu Shen, Sanqiang Zhao, Xiao Zhou, Pengwei Hu, Randy Zhong, Cheng Niu, Jie Zhou. 2020

47 citations recorded by the source corpus.

Neural network-based sequence-to-sequence (seq2seq) models strongly suffer from the low-diversity problem when it comes to open-domain dialogue generation. As bland and generic utterances usually dominate the frequency distribution in our daily chitchat, avoiding them to generate more interesting responses requires complex data filtering, sampling techniques or modifying the training objective. In this paper, we propose a new perspective to diversify dialogue generation by leveraging non-conversational text. Compared with bilateral conversations, non-conversational text are easier to obtain, more diverse and cover a much broader range of topics. We collect a large-scale non-conversational corpus from multi sources including forum comments, idioms and book snippets. We further present a training paradigm to effectively incorporate these text via iterative back translation. The resulting model is tested on two conversational datasets from different domains and is shown to produce significantly more diverse responses without sacrificing the relevance with context.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pascale Fung
2. Andrea Madotto
3. Tom Young
4. Wenliang Dai
5. Nayeon Lee
6. Yan Xu
7. Etsuko Ishii
8. Ziwei Ji

In Natural Language Processing

1. Pascale Fung
2. Nayeon Lee
3. Andrea Madotto
4. Etsuko Ishii
5. Yejin Bang
6. Yan Xu
7. Rita Frieske
8. Dan Su

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, background): “It is common for NLG tasks to have multiple plausible outputs from the same input, which is known as one-to-many mapping [64, 173].”
• Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Likewise, Su et al. (2020) incorporated external knowledge sources to diversify the response generation, but the difference was that they utilized non-conversational texts like news articles as relevant knowledge facts, which were obviously easier to obtain.”
• CoLV: A Collaborative Latent Variable Model for Knowledge-Grounded Dialogue Generation (2021, background): “Su et al. (2020) proposed to augment the dialogue generation by utilizing external non-conversational text, which is effective but also introduce noise.”