DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks

By Semih Yavuz, Abhinav Rastogi, Guan-Lin Chao, Dilek Z. Hakkani-Tür. 2019

81 citations recorded by the source corpus.

Recent advances in neural sequence-to-sequence models have led to promising results for several language generation-based tasks, including dialogue response generation, summarization, and machine translation. However, these models are known to have several problems, especially in the context of chit-chat based dialogue systems: they tend to generate short and dull responses that are often too generic. Furthermore, these models do not ground conversational responses on knowledge and facts, resulting in turns that are not accurate, informative and engaging for the users. In this paper, we propose and experiment with a series of response generation models that aim to serve in the general scenario where in addition to the dialogue context, relevant unstructured external knowledge in the form of text is also assumed to be available for models to harness. Our proposed approach extends pointer-generator networks (See et al., 2017) by allowing the decoder to hierarchically attend and copy from external knowledge in addition to the dialogue context. We empirically show the effectiveness of the proposed model compared to several baselines including (Ghazvininejadet al., 2018; Zhang et al., 2018) through both automatic evaluation metrics and human evaluation on ConvAI2 dataset.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “The PersonaChat datasets [31, 228] have accelerated research into persona consistency [68, 91, 126, 208, 219, 224, 232].”
Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021, background): “Yavuz et al. (2019) While the above approaches and models focus on incorporating knowledge and context to generate responses, they do that at the sentence or paragraph level.”
A Survey of Knowledge-enhanced Text Generation (2020, methodology): “length of grounded document. attention [80, 81, 84], topic attention [90, 152, 158, 174], knowledge base attention [44, 60, 144], knowledge graph attention [65, 75, 173, 183], grounded text attention [11, 96, 121, 169] and etc. 2.2.2 Copy and Pointing Mechanisms. Copy and pointing mechanisms are use”
Generating Persona Consistent Dialogues by Exploiting Natural Language Inference (2019, methodology): “Following this line, Yavuz et al. (2018) apply the DeepCopy model in the persona-based dialogue generation.”
Open-book Video Captioning with Retrieve-Copy-Generate Network (2021, background): “DeepCopy [32] also retrieves relevant unstructured sentences as external knowledge to assist dialog generation.”
A Controllable Model of Grounded Response Generation (2020, background): “Other researchers propose a copy mechanism to import tokens from dialogue context and grounding (Yavuz et al. 2019) or leverage a reading comprehension model to co-encode dialogue context and grounding (Qin et al. 2019”
Policy-Driven Neural Response Generation for Knowledge-Grounded Dialog Systems (2020, background): “Ghazvininejad et al. (2018), Yavuz et al. (2019), and Zhou et al. (2018) used end-to-end memory networks, copy mechanisms and static graph attention mechanisms respectively to incorporate knowledge.”
Low-Resource Knowledge-Grounded Dialogue Generation (2020, background): “, 2016) and copying (See et al., 2017; Raghu et al., 2019; Yavuz et al., 2019), but is unique in that components are pre-trained from various sources, thanks to the disentangled design.”
Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation (2021, background): “Description of the annotation for these labels are in Section 3.1. neural response generation work proposed grounding on knowledge sentences that are relevant to the dialog context (Ghazvininejad et al., 2018; Yavuz et al., 2019; Zhou et al., 2018; Roller et al., 2020b).”
Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue (2020, background): “…collecting new datasets (Qin et al., 2019; Zhang et al., 2018; Ghazvininejad et al., 2018; Zhou et al., 2018; Dinan et al., 2019; Moghe et al., 2018) or developing new models (Lian et al., 2019; Li et al., 2019b; Yavuz et al., 2019; Zhao et al., 2019b; Dinan et al., 2019; Liu et al., 2019).”