Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems

By Liu Yang, Minghui Qiu, Chen Qu, Jiafeng Guo, Yongfeng Zhang, W. Bruce Croft, Jun Huang, Haiqing Chen. 2018

149 citations recorded by the source corpus.

Intelligent personal assistant systems with either text-based or voice-based conversational interfaces are becoming increasingly popular around the world. Retrieval-based conversation models have the advantages of returning fluent and informative responses. Most existing studies in this area are on open domain ''chit-chat'' conversations or task / transaction oriented conversations. More research is needed for information-seeking conversations. There is also a lack of modeling external knowledge beyond the dialog utterances among current conversational models. In this paper, we propose a learning framework on the top of deep neural matching networks that leverages external knowledge for response ranking in information-seeking conversation systems. We incorporate external knowledge into deep neural models with pseudo-relevance feedback and QA correspondence knowledge distillation. Extensive experiments with three information-seeking conversation data sets including both open benchmarks and commercial data show that, our methods outperform various baseline methods including several deep text matching models and the state-of-the-art method on response selection in multi-turn conversations. We also perform analysis over different response types, model variations and ranking examples. Our models and research findings provide new insights on how to utilize external knowledge with deep neural models for response selection and have implications for the design of the next generation of information-seeking conversation systems.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring (2019, methodology): “, 2017), Deep Matching Networks (Yang et al., 2018), Gated Self-Attention (Zhang et al.”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “[166] extended SMNwith external knowledge in information-seeking conversation systems.”
Continuous and Interactive Factual Knowledge Learning in Veriﬁcation Dialogues (2020, background): “Recently, many conversational systems have been proposed, e.g., Conversational Recommender Systems [46, 1, 58] and Information-seeking Conversation [55, 40].”
Towards Conversational Search and Recommendation: System Ask, User Respond (2018, background): “Yang et al [41, 42] conducted next question prediction and response ranking in conversations.”
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring (2019, background): “, 2017), Deep Matching Networks (Yang et al., 2018), Gated Self-Attention (Zhang et al.”
A Deep Look into Neural Ranking Models for Information Retrieval (2019, methodology): “external knowledge [29, 30], and other novel uses of neural approaches for IR tasks [31, 32].”
Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search (2020, methodology): “[57] was developed to use knowledge bases as external resources for response ranking in conversation.”
Lifelong and Interactive Learning of Factual Knowledge in Dialogues (2019, background): “, 2018), information-seeking conversational agents (Yang et al., 2018), etc.”
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching (2020, unknown): “As there is rapid progress of representation learning with neural networks, previous research has explored deep learning based semantic matching models for paraphrase identification [38], answer selection [28, 34], and conversation response ranking [32, 35].”
Open-Retrieval Conversational Question Answering (2020, methodology): “A significant limitation of this setting is that an answer is either extracted from a given passage [34] or selected from a given candidate set [53].”
Real-time Inference in Multi-sentence Tasks with Deep Pretrained Transformers (2019, methodology): “This has been explored with Sequential Matching Network CNN-based architectures (Wu et al., 2016), Deep Matching Networks (Yang et al., 2018), Gated Self-Attention (Zhang et al., 2018b), and most recently transformers (Wolf et al., 2019; Vig and Ramea, 2019; Urbanek et al., 2019).”
CAsT-19: A Dataset for Conversational Information Seeking (2020, background): “[10] present a recent system that uses a neural matching network, pseudo relevance feedback, and neural question answering techniques to”
Answer Interaction in Non-factoid Question Answering Systems (2019, background): “Finally, conversational recommendation [40] and response ranking [36] have been explored under this multi-turn interaction setting.”