Chain-of-Verification Reduces Hallucination in Large Language Models

By Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston. Annual Meeting of the Association for Computational Linguistics, 2023

515 citations recorded by the source corpus.

Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models. We study the ability of language models to deliberate on the responses they give in order to correct their mistakes. We develop the Chain-of-Verification (CoVe) method whereby the model first (i) drafts an initial response; then (ii) plans verification questions to fact-check its draft; (iii) answers those questions independently so the answers are not biased by other responses; and (iv) generates its final verified response. In experiments, we show CoVe decreases hallucinations across a variety of tasks, from list-based questions from Wikidata, closed book MultiSpanQA and longform text generation.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/4b0b56be0ae9479d2bd5c2f0943db1906343c10f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023, methodology): “Within four months since its initial release, FA CT S CORE has actively been used in subsequent work, evaluating factual precision of recently-proposed models (Ye et al., 2023; Sun et al., 2023; Malaviya et al., 2023; Dhuliawala et al., 2023).”
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (2024, unknown): “While Chain-of-Thought prompting enhances reasoning in LLMs, methods like Auto-CoT, which automate demonstration generation, face challenges from misleading similarity (incorrect rationales in similar examples) and ineffective diversity (irrelevant or overly varied patterns).”
Toward expert-level medical question answering with large language models (2025, unknown): “It is distinct from retrieval-augmented generation approaches that leverage a fixed corpus and embedding space to find documents to condition LLM generations on 58 , and is most similar to other approaches that break down verification of claims into multiple steps 59,60 .”
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (2023, methodology): “We also compare our method to concurrent work, CoVE 65 B (Dhuliawala et al., 2023), which introduces iterative prompt engineering to improve the factuality of LLM generations.”
Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering (2024, background): “Due to the unique nature of code generation tasks, common prompting techniques that have been optimized for natural language tasks [4, 13, 10], may not be as effective when applied to code generation.”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “Dhuliawala et al. (2023) develop a decoding framework known as the Chain-of-Verification (C O V E ).”
LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked (2023, background): “This falls in line with their ability to verify the content they generate for accuracy and then correct themselves using a chain of verification, which has been shown to improve factual question answering data [6], mathematical reasoning [19], and logical reasoning [29].”
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review (2023, background): “…showed that GPT-3 can reduce hallucinations by studying various components of architectures such as Retrieval Augmented Generation (RAG) [73], Fusion-inDecoder (FiD) [74] and Seq2seq [75–77], etc. [78] developed Chain-of-Verification (COVE) to reduce the hallucinations across a variety of tasks.”
Retrieval-Augmented Generation for Large Language Models: A Survey (2023, background): “Alignment optimization addresses alignment issues and disparities between documents by introducing “hypothetical questions” [ Li et al. , 2023d ] into documents to rectify alignment issues and differences.”
Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024, background): “Chain-of-Verification, introduced by Dhuliawala et al. [13], prompts the LLM to self-correct their mistakes during generation.”