Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances

By Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, Jie Zhou. 2021

67 citations recorded by the source corpus.

Nowadays, open-domain dialogue models can generate acceptable responses according to the historical context based on the large-scale pre-trained language models. However, they generally concatenate the dialogue history directly as the model input to predict the response, which we named as the flat pattern and ignores the dynamic information flow across dialogue utterances. In this work, we propose the DialoFlow model, in which we introduce a dynamic flow mechanism to model the context flow, and design three training objectives to capture the information dynamics across dialogue utterances by addressing the semantic influence brought about by each utterance in large-scale pre-training. Experiments on the multi-reference Reddit Dataset and DailyDialog Dataset demonstrate that our DialoFlow significantly outperforms the DialoGPT on the dialogue generation task. Besides, we propose the Flow score, an effective automatic metric for evaluating interactive human-bot conversation quality based on the pre-trained DialoFlow, which presents high chatbot-level correlation (r=0.9) with human ratings among 11 chatbots. Code and pre-trained models will be public.

Subfields: Artificial Intelligence, Natural Language Processing, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Dialogue & Conversational AI

Citations in the corpus

FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation (2022, background): “However, they focus only on a single dimension, such as coherence or overall impression (Mesgar et al., 2020; Zhang et al., 2021a; Li et al., 2021; Ghazarian et al., 2022).”
Pre-Trained Language Models for Text Generation: A Survey (2022, methodology): “[112] employed a Transformer to encode each utterance into a dense vector, upon which a left-to-right flow module was designed to capture the utterance-level dynamic information flow.”
What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation (2022, background): “It is also important to get some holistic evaluation at the dialog level in order to assess the dialogs as a whole (Zhang et al., 2021a; Li et al., 2021; Mehri and Eskenazi, 2020; Finch et al., 2021).”