Sequential Match Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots

By Yu Wu, Wei Yu Wu, Ming Zhou, Zhoujun Li. 2016

508 citations recorded by the source corpus.

We study response selection for multi-turn conversation in retrieval based chatbots. Existing works either ignores relationships among utterances, or misses important information in context when matching a response with a highly abstract context vector finally. We propose a new session based matching model to address both problems. The model first matches a response with each utterance on multiple granularities, and distills important matching information from each pair as a vector with convolution and pooling operations. The vectors are then accumulated in a chronological order through a recurrent neural network (RNN) which models the relationships among the utterances. The final matching score is calculated with the hidden states of the RNN. Empirical study on two public data sets shows that our model can significantly outperform the state-of-the-art methods for response selection in multi-turn conversation.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring (2019, methodology): “This has been explored with Sequential Matching Network CNN-based architectures (Wu et al., 2017), Deep Matching Networks (Yang et al., 2018), Gated Self-Attention (Zhang et al., 2018b), and most recently transformers (Wolf et al., 2019; Vig & Ramea, 2019; Urbanek et al., 2019).”
The Second Conversational Intelligence Challenge (ConvAI2) (2019, methodology): “A detailed description of our “wild” dataset and a comparison with the data from the ﬁrst ConvAI competition will be described in detail in separate publication 9 .”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, methodology): “lenges in Building Intelligent Open-domain Dialog Systems 7 (CNN). Afterwards, the feature maps are transformed with multiple CNN layers to form the final representation for classification. Wu et al. [106] proposed a sequential matching model for multi-turn dialog where each contextual utterance in ”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, background): “ned them with current messages to form a reformulated context. [124] performed context-response matching on not only the general word level context vector but also the utterance level context vector. [106] further improved the leveraging of utterances relationship and contextual information by match”
Response Generation by Context-aware Prototype Editing (2018, methodology): “Prior works on retrieval-based methods mainly focus on the matching model architecture for single turn conversation (Hu et al. 2014) and multi-turn conversation (Lowe et al. 2015; Zhou et al. 2016; Wu et al. 2017).”
MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation (2022, background): “As advanced works on dialogue systems include retrieval-based methods (Wu et al., 2017; Zhou et al., 2018b; Whang et al., 2020; Li et al., 2021) and generative methods (Li et al., 2016; Serban et al., 2016; Zhang et al., 2020).”
A taxonomy, data set, and benchmark for detecting and classifying malevolent dialogue responses (2021, background): “There are several data sets related to multiturn dialogues, that is, Ubuntu (Lowe et al., 2015), DailyDialog (Li et al., 2017), Douban (Wu et al., 2017), and E-commerce (Z. Zhang et al., 2018), but they are not for malevolent dialogue evaluation.”
Diversifying Dialogue Generation with Non-Conversational Text (2020, methodology): “Standard retrieval algorithms based on context-matching (Wu et al., 2017; Bartl and Spanakis, 2017) fail to apply here since non-conversational text does not come with its corresponding context.”
ConveRT: Efficient and Accurate Conversational Representations from Transformers (2019, unknown): “Response selection is also directly applicable to retrieval-based dialog systems, a popular and elegant approach to framing dialog (Wu et al., 2017; Weston et al., 2018; Mazaré et al., 2018; Gunasekara et al., 2019; Henderson et al., 2019b).”
Personalized Reason Generation for Explainable Song Recommendation (2019, background): “One approach to this is retrieval-based [55, 56]: match a user input with existing question-and-answer pairs to retrieve the most appropriate responses.”
You Impress Me: Dialogue Generation via Mutual Persona Perception (2020, background): “Retrieval-based meth-ods retrieve response candidates and rank them based on the matching scores with the dialogue (Sordoni et al., 2015; Wu et al., 2017; Gu et al., 2019).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Some works built retrieval-based dialogue systems (Wu et al., 2016; Feng et al., 2019; Tao et al., 2019; Lu et al., 2019).”
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network (2018, methodology): “RNN-based models : Previous best performing models are based on RNNs, we choose representative models as baselines, including SMNdynamic(Wu et al., 2017), Multiview(Zhou et al., 2016), DualEncoderlstm and DualEncoderbilstm (Lowe et al., 2015), DL2R (Yan et al., 2016), Match-LSTM (Wang and Jiang, 201”
A Repository of Conversational Datasets (2019, background): “This measures how often the correct response is identified as one of the top k ranked responses (Lowe et al., 2015; Inaba and Takahashi, 2016; Yu et al., 2016; Al-Rfou et al., 2016; Henderson et al., 2017; Lowe et al., 2017; Wu et al., 2017; Cer et al., 2018; Chaudhuri et al., 2018; Du and Black, 20”
Building Sequential Inference Models for End-to-End Response Selection (2018, methodology): “2016) and retrieval-based methods (Zhou et al. 2016; Wu et al. 2017).”
Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus (2018, methodology): “Douban conversation corpus (Wu et al., 2017) which are constructed from Douban group 3 (a popular social networking service in China) is also used in experiments.”