On Hallucination and Predictive Uncertainty in Conditional Language Generation

By Yijun Xiao, William Yang Wang. 2021

264 citations recorded by the source corpus.

Despite improvements in performances on different natural language generation tasks, deep neural models are prone to hallucinating facts that are incorrect or nonexistent. Different hypotheses are proposed and examined separately for different tasks, but no systematic explanations are available across these tasks. In this study, we draw connections between hallucinations and predictive uncertainty in conditional language generation. We investigate their relationship in both image captioning and data-to-text generation and propose a simple extension to beam search to reduce hallucination. Our analysis shows that higher predictive uncertainty corresponds to a higher chance of hallucination. Epistemic uncertainty is more indicative of hallucination than aleatoric or total uncertainties. It helps to achieve better results of trading performance in standard metric for less hallucination with the proposed beam search variant.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Generative Models

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “[174], Xiao and Wang [211] Puduppully and Lapata [148]”
SafeText: A Benchmark for Exploring Physical Safety in Language Models (2022, unknown): “(Xiao and Wang, 2021; Gehrmann et al., 2022; Ji et al., 2022).”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, background): “However, high certainty does not mean truthful [133, 134, 75], these methods suffer from poor calibration of LLMs [131, 16], difficulty in evaluating free-form text [76], and poor interpretability.”
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023, methodology): “Additionally, uncertainty estimation based on aleatoric and epistemic uncertainty for autoregressive generation has been studied (Xiao and Wang, 2021; Malinin and Gales, 2021), though these methods require access to full output distributions.”
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (2023, background): “…can provide valuable insights into the reliability of its responses, facilitating risk assessment and error mitigation (Kuleshov et al., 2018; Kuleshov & Deshpande, 2022), selective generation (Ren et al., 2022), and reducing hallucinations in natural language generation tasks (Xiao & Wang, 2021).”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “Conventional NLG models are typically designed for a single task, and thus, hallucination studies on them are usually task-specific (Maynez et al., 2020; Wang and Sennrich, 2020; Xiao and Wang, 2021); however, current LLMs are expected to excel in multi-task, multi-lingual, and multi-domain settings”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “Xiao and Wang (2021) Prompting-based Metrics.”
Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation (2021, background): “Thus, from another point of view, functions applied on the probability distribution are also suitable to approximate Yx,k − Yx∗,k, such as the perplexity and uncertainty (Xu et al., 2020; Xiao and Wang, 2021).”
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2023, background): “But in the con-320 text of this paper, which focuses on the relatively strict RAG scenarios, we have labeled information that is not mentioned in the reference but may be truthful as hallucinations.”
Detecting hallucinations in large language models using semantic entropy (2024, unknown): “We measure entailment using both general-purpose LLMs and natural language inference (NLI) tools developed specifically for detecting entailment for which we show direct evaluations in Supplementary Tables 2 and 3 and Supplementary Fig.”
LLM-confidence reranker: A training-free approach for enhancing retrieval-augmented generation systems (2026, unknown): “Xiao & Wang (2021) further applied entropy values to the hallucination problem in conditional language generation tasks, revealing the key relationship between uncertainty and hallucination.”