A Knowledge-Grounded Neural Conversation Model
By Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, William B. Dolan, Jianfeng Gao, Wen-tau Yih, Michel Galley. 2017
591 citations recorded by the source corpus.
Neural network models are capable of generating extremely natural sounding conversational interactions. However, these models have been mostly applied to casual scenarios (e.g., as “chatbots”) and have yet to demonstrate they can serve in more useful conversational applications. This paper presents a novel, fully data-driven, and knowledge-grounded neural conversation model aimed at producing more contentful responses. We generalize the widely-used Sequence-to-Sequence (Seq2Seq) approach by conditioning responses on both conversation history and external “facts”, allowing the model to be versatile and applicable in an open-domain setting. Our approach yields significant improvements over a competitive Seq2Seq baseline. Human judges found that our outputs are significantly more informative.
Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Jason Aaron Edward Weston
- Yejin Choi
- Luke S. Zettlemoyer
- Percy Liang
- Noah A. Smith
- Hannaneh Hajishirzi
In Retrieval-Augmented Generation
- Jason Aaron Edward Weston
- Hannaneh Hajishirzi
- Sebastian Riedel
- Arthur Szlam
- Wen-tau Yih
- Jianfeng Gao
- Fabio Petroni
- Kurt Shuster
In Natural Language Processing
- Jason Aaron Edward Weston
- Jianfeng Gao
- Mohit Bansal
- Yejin Choi
- Luke S. Zettlemoyer
- Noah A. Smith
- Hannaneh Hajishirzi
- Percy Liang
Citations in the corpus
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, background): “Other work improves the ability of dialog models to generate factual text by attending over fact embeddings [9, 13] or, closer to our work, over retrieved text directly [15].”
- QuAC: Question Answering in Context (2018, background): “19 Dialog fits into an increasing interest in open domain dialog, mostly studied in the context of social chit-chat (Li et al., 2016; Ritter et al., 2011; Fang et al., 2017; Ghazvininejad et al., 2018).”
- Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, background): “If the agent has been trained to retrieve and incorporate information from an external source of knowledge (Dinan et al., 2019b; Qin et al., 2019b; Prabhumoye, Quirk, and Galley, 2019; Ghazvininejad et al., 2018), then simply updating that source would allow the agent to inject current information i”
- Wizard of Wikipedia: Knowledge-Powered Conversational agents (2018, methodology): “Both Ghazvininejad et al. (2018) and Parthasarathi & Pineau (2018) use unstructured text instead, as we do: the former to discuss news articles using Wikipedia summaries as knowledge, and the latter to discuss local businesses in two-turn dialogues using Foursquare tips as knowledge.”
- Towards Exploiting Background Knowledge for Building Conversation Systems (2018, methodology): “While there are already some works in this direction (Rojas-Barahona et al., 2017; Williams et al., 2016; Lowe et al., 2015a; Ghazvininejad et al., 2017) which try to integrate external knowledge sources with existing datasets, we believe that building new datasets where the utterances are…”
- Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “becoming an important topic, with several datasets proposed that attempt to model its occurrence (Dinan et al., 2019b; Ghazvininejad et al., 2018; Gopalakrishnan et al., 2019; Galetzka et al., 2020).”
- Internet-Augmented Dialogue Generation (2021, background): “In dialogue, knowledge grounding is becoming more popular an area, with several datasets developed to study it (Zhou et al., 2018; Dinan et al., 2019; Ghazvininejad et al., 2018; Gopalakrishnan et al., 2019; Galetzka et al., 2020).”
- Neural Approaches to Conversational AI (2018, positive): “Only recently have researchers begun to explore how to ground the chitchat in world knowledge (Ghazvininejad et al., 2018) and images (Mostafazadeh et al.”
- Training Millions of Personalized Dialogue Agents (2018, background): “Other classic strategies include extracting explicit variables from structured knowledge bases or other symbolic sources as in (Ghazvininejad et al., 2017; Joshi et al., 2017; Young et al., 2017).”
- Controlling Style in Generated Dialogue (2020, background): “…have explored ways to give generation stable grounding (e.g., a persona (Zhang et al., 2018; Dinan et al., 2020), knowledge (Dinan et al., 2019; Ghazvininejad et al., 2018), personal situations (Rashkin et al., 2019), internet websites (Keskar et al., 2019), previous conversations from same…”
- Challenges in Building Intelligent Open-domain Dialog Systems (2019, background): “Ct can also encode other (non-content) contexts such as persona [103, 175, 186] for personalized dialog generation, emotion labels [4, 189] for emotional response generation, and knowledge graphs [36, 190] for knowledge-aware response generation.”
- Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading (2019, methodology): “, 2018a), review posts (Ghazvininejad et al., 2018; Moghe et al., 2018), and even images (Das et al.”
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention (2021, background): “, 2020a), dialog response generation (Ghazvininejad et al., 2018), conversational QA (Qin et al.”
- Extending Neural Generative Conversational Model using External Knowledge Sources (2018, methodology): “Similarly, instead of dynamically querying the web, (Ghazvininejad et al., 2017) pre-trains the model from facts in Foursquare (Yang et al.”
- A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, background): “[22] made attempts on top of this and has achieved good performance in open-domain conversations.”
- Graph Constrained Reinforcement Learning for Natural Language Action Spaces (2020, background): “Ghazvininejad et al. (2017) and Guan et al. (2018) effectively use knowledge graph representations to improve neural conversational and story ending prediction models respectively.”
- Commonsense Knowledge Aware Conversation Generation with Graph Attention (2018, methodology): “Several studies incorporated unstructured texts as external knowledge into conversation generation [ Ghazvininejad et al. , 2017; Long et al. , 2017].”
- Retrieval-guided Dialogue Response Generation via a Matching-to-Generation Framework (2019, background): “Various approaches (Li et al., 2016b; Xing et al., 2017; Ghazvininejad et al., 2018; Zhou et al., 2018; Liu et al., 2018; Bi et al., 2019; Tian et al., 2019; Gao et al., 2019) have been proposed for this problem.”