Neural Responding Machine for Short-Text Conversation

By Lifeng Shang, Zhengdong Lu, Hang Li. 2015

1,163 citations recorded by the source corpus.

We propose Neural Responding Machine (NRM), a neural network-based response generator for Short-Text Conversation. NRM takes the general encoder-decoder framework: it formalizes the generation of response as a decoding process based on the latent representation of the input text, while both encoding and decoding are realized with recurrent neural networks (RNN). The NRM is trained with a large amount of one-round conversation data collected from a microblogging service. Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Semi-supervised Sequence Learning (2015, methodology): “[32], which has been successfully used for machine translation [21, 11], text parsing [33], image captioning [35], video analysis [31], speech recognition [4] and conversational modeling [28, 34].”
MASS: Masked Sequence to Sequence Pre-training for Language Generation (2019, background): “…some inputs, including tasks like neural machine translation (NMT) (Cho et al., 2014; Bahdanau et al., 2015a; Vaswani et al., 2017), text summarization (Ayana et al., 2016; Suzuki & Nagata, 2017; Gehring et al., 2017) and conversational response generation (Shang et al., 2015; Vinyals & Le, 2015).”
Unsupervised Pretraining for Sequence to Sequence Learning (2016, background): “Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 383–391 Copenhagen, Denmark, September 7–11, 2017. c©2017 Association for Computational Linguistics”
LaMDA: Language Models for Dialog Applications (2022, background): “Dialog models [14, 15, 16], one of the most interesting applications of large language models, successfully take advantage of Transformers’ ability to represent long-term dependencies in text [17, 18].”
Towards a Human-like Open-Domain Chatbot (2020, background): “End-to-end neural network approaches (Shang et al., 2015; Vinyals and Le, 2015; Sordoni et al., 2015; Serban et al., 2016; Zhang et al., 2019), on the other hand, offer the simplicity of a single learned model.”
Towards Exploiting Background Knowledge for Building Conversation Systems (2018, background): “, 2015) leading to the development of several end-to-end conversation systems (Shang et al., 2015; Vinyals and Le, 2015; Li et al., 2016; Serban et al., 2016) which treat dialog as a sequence generation task.”
A Knowledge-Grounded Neural Conversation Model (2017, methodology): “This paradigm was introduced by models (Sordoni et al., 2015; Serban et al., 2016; Shang et al., 2015; Vinyals and Le, 2015; Li et al., 2016a,b).”
A Diversity-Promoting Objective Function for Neural Conversation Models (2015, background): “…of statistical machine translation (SMT), either phrase-based (Ritter et al., 2011), or using neural networks to rerank, or directly in the form of sequence-to-sequence (S EQ 2S EQ ) models (Sordoni et al., 2015; Shang et al., 2015; Vinyals and Le, 2015; Wen et al., 2015; Serban et al., 2016).”
Does Gender Matter? Towards Fairness in Dialogue Systems (2019, background): “Dialogue systems are often built based on real human conversational data through machine learning especially deep learning techniques [12, 13, 14].”
Towards Coherent and Cohesive Long-form Text Generation (2018, background): “els based on Recurrent Neural Networks (RNNs) are widely applied to response generation for dialogue (Vinyals and Le, 2015; Shang et al., 2015; Sordoni et al., 2015; Li et al., 2015).”
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, background): “Neural conversational models are effective in generating coherent and relevant responses [1, 2, 3, 4, etc.].”
A Neural Conversational Model (2015, background): “(Shang et al., 2015), used recurrent neural networks to model dialogue in short conversations (trained on Twitter-style chats).”
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (2015, background): “Most recently Sor-doni et al. (2015b) and Shang et al. (2015) have drawn in-spiration from the use of neural networks in natural language modeling and machine translation tasks (Cho et al. 2014).”
Deep Reinforcement Learning for Dialogue Generation (2016, background): “Neural response generation (Sordoni et al., 2015; Shang et al., 2015; Vinyals and Le, 2015; Li et al., 2016a; Wen et al., 2015; Yao et al., 2015; Luan et al., 2016; Xu et al., 2016; Wen et al., 2016; Li et al., 2016b; Su et al., 2016) is of growing interest.”
Neural Approaches to Conversational AI (2018, neutral): “Together with the contemporaneous works (Shang et al., 2015; Vinyals and Le, 2015), these papers presented the first neural approaches to fully E2E conversation modeling.”