Response Generation by Context-aware Prototype Editing

By Yu Wu, Furu Wei, Shaohan Huang, Zhoujun Li, Ming Zhou. 2018

128 citations recorded by the source corpus.

Open domain response generation has achieved remarkable progress in recent years, but sometimes yields short and uninformative responses. We propose a new paradigm, prototypethen-edit for response generation, that first retrieves a prototype response from a pre-defined index and then edits the prototype response according to the differences between the prototype context and current context. Our motivation is that the retrieved prototype provides a good start-point for generation because it is grammatical and informative, and the post-editing process further improves the relevance and coherence of the prototype. In practice, we design a contextaware editing model that is built upon an encoder-decoder framework augmented with an editing vector. We first generate an edit vector by considering lexical differences between a prototype context and current context. After that, the edit vector and the prototype response representation are fed to a decoder to generate a new response. Experiment results on a large scale dataset demonstrate that our new paradigm significantly increases the relevance, diversity and originality of generation results, compared to traditional generative models. Furthermore, our model outperforms retrieval-based methods in terms of relevance and originality.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

A Retrieve-and-Edit Framework for Predicting Structured Outputs (2018, background): “Recent work on dialogue generation [40, 30, 37] proposed a joint system in which an RNN is trained to transform a retrieved candidate.”
What Makes Good In-Context Examples for GPT-3? (2021, background): “…al., 2020; Mao et al., 2020), dialogue generation (Yan et al., 2016; Cai et al., 2018; Song et al., 2016; Pandey et al., 2018; We-ston et al., 2018; Wu et al., 2019), text summarization (Cao et al., 2017; Peng et al., 2019), data-to-text generation (Peng et al., 2019), and text-to-code generation…”
Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “However, as pointed out in [75], ADEM can be easily fooled with a variation as simple as reversing the word order in the text.”
Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing (2019, methodology): “Retrieval-augmented models have also been studied in text generation (Gu et al., 2017; Huang et al., 2018b; Guu et al., 2018; Wu et al., 2018).”
Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation (2019, neutral): “…related to the recent work on applying retrieval mechanisms to augment text generation, such as image captioning (Kuznetsova et al., 2013; Mason and Charniak, 2014), dialogue generation (Song et al., 2016; Yan et al., 2016; Wu et al., 2018) and style transfer (Lin et al., 2017; Li et al., 2018).”
Controlling Text Edition by Changing Answers of Specific Questions (2021, background): “Examples are the generation of summaries by template-based rewriting (Cao et al., 2018; Hashimoto et al., 2018) and the generation of text or a response by editing a prototype sentence (Guu et al., 2018; Pandey et al., 2018; Wu et al., 2019).”
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents (2022, background): “Some issues are that they may not generalize as well to data beyond that which they are trained (overfit) and also may be biased and gameable (Wu et al., 2019; Albrecht and Hwa, 2007).”
Learning to Represent Edits (2018, methodology): “While these models typically require an “editor” component to generate the output by exploiting the difference between similar inputs, they usually use the simpler bag-of-edits representations (Wu et al., 2019), or implicitly capture it via end-to-end neural networks (Contractor et al.”
Text Editing by Command (2020, background): “Wu et al. (2019) expand upon Guu et al. (2018)’s setting, but for dialog.”
Retrieval-guided Dialogue Response Generation via a Matching-to-Generation Framework (2019, methodology): “• EditVec The model proposed in Wu et al. (2019).”
Template Guided Text Generation for Task Oriented Dialogue (2020, methodology): “Our template based approach bears similarities to sentence fusion (Barzilay and McKeown, 2005), and prototype based text editing (Hossain et al., 2020; Cao et al., 2018; Guu et al., 2018; Wu et al., 2019).”