Modeling Multi-turn Conversation with Deep Utterance Aggregation

By Zhuosheng Zhang, Jiangtong Li, Peng Fei Zhu, Zhao Hai, Gongshen Liu. 2018

256 citations recorded by the source corpus.

Multi-turn conversation understanding is a major challenge for building intelligent dialogue systems. This work focuses on retrieval-based response matching for multi-turn conversation whose related work simply concatenates the conversation utterances, ignoring the interactions among previous utterances for context modeling. In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context representation. In detail, a self-matching attention is first introduced to route the vital information in each utterance. Then the model matches a response with each refined utterance and the final matching score is obtained after attentive turns aggregation. Experimental results show our model outperforms the state-of-the-art methods on three multi-turn conversation benchmarks, including a newly introduced e-commerce dialogue corpus.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, background): “Machine reading comprehension (MRC) is a fundamental and long-standing goal of natural language understanding which aims to teach machine to answer question automatically according to given passage (Hermann et al. 2015; Ra-jpurkar et al. 2016; Nguyen et al. 2016; Zhang et al. 2018).”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, methodology): “[181] proposed a deep utterance aggregation model which shares a similar structure with SMN.”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, background): “Visual Dialogue; Speaker Consistency and Personality Response Shuster et al. (2018)”
FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning (2021, unknown): “We use 11 baselines for comparison, including the traditional TF-IDF (Paik, 2013), Dual LSTM (Lowe et al., 2015), SMN (Wu et al., 2017), DAM (Zhou et al., 2018), BERT (Devlin et al., 2019), BERT-MC (Cui et al., 2020), GPT-2 (Radford et al., 2019), Deep Utterance Aggregation (DUA) (Zhang et al., 2018”
Effective Character-Augmented Word Embedding for Machine Reading Comprehension (2018, background): “Machine reading comprehension (MRC) is a challenging task which requires computers to read and understand documents to answer corresponding questions, it is indispensable for advanced context-sensitive dialogue and interactive systems [12,34,36].”
MuTual: A Dataset for Multi-Turn Dialogue Reasoning (2020, background): “Owing to the rise of deep learning techniques and the large amount of conversation data for training (Lowe et al., 2015; Wu et al., 2017; Zhang et al., 2018b), we are now witnessing promising results of chatbots both in academia and industry (Pan et al., 2019; Tao et al., 2019).”
Semantics-aware BERT for Language Understanding (2019, background): “A common trend among NLU studies is that models are becoming more and more sophisticated with stacked attention mechanisms or large amount of corpus (Zhang et al. 2018; 2020a; Zhou, Zhang, and Zhao 2019), resulting in explosive growth of computational cost.”
Effective Subword Segmentation for Text Comprehension (2018, background): “Distributed word representation plays a fundamental role in neural models (Zhang et al. 2018a; Wang et al. 2018; Zhang et al. 2018b).”
Real-time Inference in Multi-sentence Tasks with Deep Pretrained Transformers (2019, methodology): “…been used to train dialogue-based models, including those curated from social media platforms (Ritter et al., 2011; Mazar´e et al., 2018), scraped from scripts or chatlogs (Tiedemann, 2012; Lowe et al., 2015), or deliberately collected via crowdsourcing (Zhang et al., 2018a; Dinan et al., 2019b).”
Sequential Attention-based Network for Noetic End-to-End Response Selection (2019, methodology): “The typical approaches for multi-turn response selection mainly consist of sequence-based methods (Lowe et al. 2015; Yan, Song, and Wu 2016) and hierarchy-based meth-ods (Zhou et al. 2016; Wu et al. 2017; Zhang et al. 2018; Zhou et al. 2018).”
Subword-augmented Embedding for Cloze Reading Comprehension (2018, background): “ep learning models have been proposed for reading comprehension (Sordoni et al., 2016; Trischler et al., 2016; Wang and Jiang, 2016; Munkhdalai and Yu, 2017; Wang et al., 2017a; Dhingra et al., 2017; Zhang et al., 2018b; Wang et al., 2018b). Notably, Chen et al. (2016) conducted an in-depth and thou”
Dual Dynamic Memory Network for End-to-End Multi-turn Task-oriented Dialog Systems (2020, methodology): “These methods introduce much noise since previous utterances as the context is lengthy and redundant (Zhang et al., 2018).”
Domain Adaptive Training BERT for Response Selection (2019, methodology): “CNN based utterance-response matching technique is applied to enhance relationships between dialog context and response (Wu et al., 2017; Zhang et al., 2018).”