Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network

By Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, Hua Wu. Annual Meeting of the Association for Computational Linguistics, 2018

349 citations recorded by the source corpus.

Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context. In this paper, we investigate matching a response with its multi-turn context using dependency information based entirely on attention. Our solution is inspired by the recently proposed Transformer in machine translation (Vaswani et al., 2017) and we extend the attention mechanism in two ways. First, we construct representations of text segments at different granularities solely with stacked self-attention. Second, we try to extract the truly matched segment pairs with attention across the context and response. We jointly introduce those two kinds of attention in one uniform neural network. Experiments on two large-scale multi-turn response selection tasks show that our proposed model significantly outperforms the state-of-the-art models.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/c91b4b3a20a7637ecbb7e0179ac3108f3cf11880

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Challenges in Building Intelligent Open-domain Dialog Systems (2019, positive): “[194] proposed a deep attention matching network.”
Sequential Attention-based Network for Noetic End-to-End Response Selection (2019, methodology): “2015; Yan, Song, and Wu 2016) and hierarchy-based methods (Zhou et al. 2016; Wu et al. 2017; Zhang et al. 2018; Wu et al. 2018).”
Relational Memory-Augmented Language Models (2022, background): “, 2015), dialogue systems (Zhou et al., 2018; Moon et al., 2019; Guo et al., 2018; Liu et al., 2021b) and recommendation systems (Zhang et al.”
Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations (2019, methodology): “, 2018a), response matching in dialogue system (Zhou et al., 2018c), language modelling (Dai et al.”
A Bichannel Transformer with Context Encoding for Document-Driven Conversation Generation in Social Media (2020, methodology): “Recently, deep neural networks have been widely used for both response retrieval [29, 30] and response generation [31].”
How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning (2019, background): “Though the retrieval-based dialogue system can be automatically evaluated using precision and recall (Zhou et al., 2018), that metrics cannot be extended for the generation-based ones.”
Domain Adaptive Training BERT for Response Selection (2019, methodology): “In addition, we especially point out comparing BERT-VFT with deep attention matching network (DAM) (Zhou et al., 2018) Wang (2019) in DSTC 7.”
MuTual: A Dataset for Multi-Turn Dialogue Reasoning (2020, methodology): “Deep Attention Matching Network : Zhou et al. (2018) adopt self attention module (Vaswani et al., 2017) to encode response and each utterance, respectively.”
ConveRT: Efficient and Accurate Conversational Representations from Transformers (2019, background): “Intuitively, the single-context assumption is limiting for modeling multi-turn conversations, where strong conversational cues can be found in earlier dialog history, and there has been a body of work on leveraging richer dialog history for response selection (Chaudhuri et al., 2018; Zhou et al., 20”
Natural Language Processing Advancements By Deep Learning: A Survey (2020, background): “response selection augmented by an attention mechanism and sequence matching [175].”
FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning (2021, positive): “We use 11 baselines for comparison, including the traditional TF-IDF (Paik, 2013), Dual LSTM (Lowe et al., 2015), SMN (Wu et al., 2017), DAM (Zhou et al., 2018), BERT (Devlin et al., 2019), BERT-MC (Cui et al., 2020), GPT-2 (Radford et al., 2019), Deep Utterance Aggregation (DUA) (Zhang et al., 2018”
A Repository of Conversational Datasets (2019, background): “This measures how often the correct response is identified as one of the top k ranked responses (Lowe et al., 2015; Inaba and Takahashi, 2016; Yu et al., 2016; Al-Rfou et al., 2016; Henderson et al., 2017; Lowe et al., 2017; Wu et al., 2017; Cer et al., 2018; Chaudhuri et al., 2018; Du and Black, 20”
MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation (2022, background): “As advanced works on dialogue systems include retrieval-based methods (Wu et al., 2017; Zhou et al., 2018b; Whang et al., 2020; Li et al., 2021) and generative methods (Li et al., 2016; Serban et al., 2016; Zhang et al., 2020).”