Retrieval Augmentation Reduces Hallucination in Conversation
By Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, Jason Aaron Edward Weston. 2021
1,126 citations recorded by the source corpus.
Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020). In this work we explore the use of neural-retrieval-in-the-loop architectures - recently shown to be effective in open-domain QA (Lewis et al., 2020b; Izacard and Grave, 2020) - for knowledge-grounded dialogue, a task that is arguably more challenging as it requires querying based on complex multi-turn dialogue context and generating conversationally coherent responses. We study various types of architectures with multiple components - retrievers, rankers, and encoder-decoders - with the goal of maximizing knowledgeability while retaining conversational ability. We demonstrate that our best models obtain state-of-the-art performance on two knowledge-grounded conversational tasks. The models exhibit open-domain conversational capabilities, generalize effectively to scenarios not within the training data, and, as verified by human evaluations, substantially reduce the well-known problem of knowledge hallucination in state-of-the-art chatbots.
Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Yejin Choi
- Hannaneh Hajishirzi
- Jason Aaron Edward Weston
- Quoc V. Le
In Retrieval-Augmented Generation
- Hannaneh Hajishirzi
- Sebastian Riedel
- Akari Asai
- Danqi Chen
- Jason Aaron Edward Weston
- Wen-tau Yih
- Matei A. Zaharia
- Christopher Potts
In Natural Language Processing
- Jason Aaron Edward Weston
- Mohit Bansal
- Yejin Choi
- Jianfeng Gao
- Luke S. Zettlemoyer
- Hannaneh Hajishirzi
- Graham Neubig
- Caiming Xiong
Citations in the corpus
- Few-shot Learning with Retrieval Augmented Language Models (2022, background): “These architectures have also been shown to reduce hallucination in dialogue agents (Shuster et al., 2021).”
- LaMDA: Language Models for Dialog Applications (2022, background): “Granting access to external knowledge bases has been shown to reduce the rate at which models hallucinate unsourced statements in dialog across a variety of retrieval systems and model architectures [31].”
- Improving language models by retrieving from trillions of tokens (2021, positive): “Overall, retrieval reduces hallucinations (in line with the findings of Shuster et al. (2021)) and makes the model more knowledgeable, when comparing with samples produced with retrieval disabled.”
- TruthfulQA: Measuring How Models Mimic Human Falsehoods (2021, background): “…Zellers et al., 2019), summarization (Gabriel et al., 2021; Maynez et al., 2020; Stiennon et al., 2020; Wang et al., 2020), conversational dialog (Shuster et al., 2021; Roller et al., 2021), and question answering (Dou et al., 2021; Krishna et al., 2021; Lewis et al., 2020; Logan IV et al.,…”
- OpenAssistant Conversations - Democratizing Large Language Model Alignment (2023, background): “Large language models are prone to generating inaccurate information about people, places, or facts, commonly known as ‘hallucinations’ [30, 31].”
- WebGPT: Browser-assisted question-answering with human feedback (2021, background): “However, prior work shows that the use of retrieval reduces the rate of hallucinations [Shuster et al., 2021], and moreover WebGPT performs about as well as human demonstrations for factual accuracy on ELI5 (see Figure 2(a)).”
- Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022, background): “Standard large language models are known to generate fluent but factually incorrect statements, a problem that is not solved by just increasing their size (Shuster et al., 2021).”
- Internet-augmented language models through few-shot prompting for open-domain question answering (2022, background): “, 2020; Izacard & Grave, 2020) that ground their decisions in external retrieved evidence to reduce hallucinations and improve factuality (Shuster et al., 2021).”
- You Only Need One Model for Open-domain Question Answering (2021, background): “Their implicit knowledge reference also makes it hard to find supporting knowledge and often results in hallucinations (Shuster et al., 2021).”
- Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, methodology): “We sort words by frequency and take the most frequent words that contribute: 50% of the probability mass toward Wizard of Wikipedia utterances (amounting to 121 words) following Shuster et al. (2021).”
- BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage (2022, methodology): “The promise of such an approach is that the distribution of data will more closely match those organic users’ desires, rather than decided by the researchers themselves when creating datasets (Gabriel et al., 2020; Roller et al., 2020; Shuster et al., 2021b; Ouyang et al., 2022).”
- PEER: A Collaborative Language Model (2022, background): “Beyond that, they are hard to control (Korbak et al., 2022) and verifying their outputs is challenging as they often hallucinate content (Maynez et al., 2020; Shuster et al., 2021; Nakano et al., 2021) and lack the ability to explain their intentions.”
- Survey of Hallucination in Natural Language Generation (2022, methodology): “Hallucination in conversations, which is also considered as a factual consistency problem, has raised much research interest recently [30, 108, 116, 121].”
- Beyond Goldfish Memory: Long-Term Open-Domain Conversation (2021, methodology): “Moreover, all SumMem-MSC models outperform their retrieval-augmented model counterparts MSC 2.7B (RAG/FiD/FiD-RAG).”
- Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, methodology): “Notably, K2R hallucinates less than any model studied by Shuster et al. (2021a).”
- Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2022, methodology): “To reduce hallucination in dialog systems, Shuster et al. (2021) introduced knowledge augmentation and Dziri et al. (2021) presented a post-processing method to refine generated outputs.”
- Tool Learning with Foundation Models (2023, neutral): “Besides, foundation models are also criticized to hallucinate knowledge (Roller et al., 2021; Shuster et al., 2021) by generating seemingly plausible but non-factual content.”
- NaturalProver: Grounded Mathematical Proof Generation with Language Models (2022, methodology): “To judge the potential utility of language models as assistive systems in natural mathematics, we measure whether generated next-steps and full proofs are potentially useful hints for proving the theorem on one’s own.”