Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models

By Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen and 5 more. 2023

1,079 citations recorded by the source corpus.

While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge. This phenomenon poses a substantial challenge to the reliability of LLMs in real-world scenarios. In this paper, we survey recent efforts on the detection, explanation, and mitigation of hallucination, with an emphasis on the unique challenges posed by LLMs. We present taxonomies of the LLM hallucination phenomena and evaluation benchmarks, analyze existing approaches aiming at mitigating LLM hallucination, and discuss potential directions for future research.

Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Safety, Robustness & Interpretability

Citations in the corpus

Multimodal Chain-of-Thought Reasoning in Language Models (2023, neutral): “However, there is severe information loss in the captioning process; thus, using the captions (as opposed to vision features) may suffer from a lack of mutual synergy in the representation space of different modalities.”
ChatGPT: The End of Online Exam Integrity? (2022, neutral): “…of limitations of Large Language Models (LLMs) like ChatGPT due to their proclivity to fabricate non-existent facts or inappropriate information [22,27–29] referred to as hallucination, and with this comes the temptation to downplay and dismiss the performance capabilities of LLMs in exam contexts.”
A Comprehensive Overview of Large Language Models (2023, neutral): “This paper will serve as a valuable resource for researchers, offering insights into the recent advancements in LLMs and providing fundamental concepts and details to develop improved LLMs.”
Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024, background): “A conventional classification of hallucination is the intrinsic-extrinsic dichotomy [14, 25, 29, 75].”
Empowering Biomedical Discovery with AI Agents (2024, positive): “Curated databases oﬀer structured and factual information, aiding in reducing the risks associated with misinformation potentially generated by agent hallucinations [43, 44].”
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, neutral): “Prior to the advent of LLMs, the NLP community has primarily defined “hallucination” as the generation of non-sensical content or content that deviates from its sources [28, 32, 39, 57, 74, 76].”
A Survey on Evaluation of Large Language Models (2023, background): “However, the information generated can include factual inaccuracies or statements ungrounded in reality, a phenomenon known as hallucination [154, 239].”
Generative Verifiers: Reward Modeling as Next-Token Prediction (2024, neutral): “On reasoning domains, LLM-based verifiers are typically trained as discriminative reward models (RMs) to assign numerical scores to candidate solutions, which is then used to classify them as correct or incorrect (Cobbe et al., 2021; Lightman et al., 2023; Wang et al., 2023).”
Agentless: Demystifying LLM-based Software Engineering Agents (2024, background): “That is to say they tend to take all information/feedback and do not know how to filter out or correct irrelevant, incorrect, or misleading information [84, 107].”
Tool learning with large language models: a survey (2024, neutral): “This inherent limitation frequently results in responses that are plausible yet factually incorrect or outdated (often referred to as hallucination) [14, 15], posing significant risks and misleading users.”
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, neutral): “Despite demonstrating remarkable reasoning (Guo et al., 2025) and coding (Guo et al., 2024) capabilities, LLMs (Zhao et al., 2023; Team, 2024; Achiam et al., 2023) often lack domain-specific knowledge (Peng et al., 2023; Li et al., 2023) and are prone to hallucinations (Zhang et al., 2023).”
Explainability for Large Language Models: A Survey (2023, background): “…of LLMs has raised concerns about their trustworthiness, as they have the potential to exhibit undesirable behaviors such as generating hallucination, a phenomenon in which models generate output that is irrelevant and nonsensical in a natural manner (Zhang et al., 2023; Huang et al., 2023).”
A Survey on Hallucination in Large Vision-Language Models (2024, background): “A growing body of literature has explored various factors contributing to hallucinations in LLMs [Ji et al. , 2023; Zhang et al. , 2023b; Huang et al. , 2023a].”
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2024, background): “However, such a post-hoc semantic measurement on decoded language sentences is inferior to precisely modeling the logical consistency/divergence Manakul et al. (2023); Zhang et al. (2023).”
Understanding the planning of LLM agents: A survey (2024, background): “Although these issues can be alleviated through careful prompt engineering, they reflect fundamental shortcomings in LLM [ Zhang et al. , 2023b; Huang et al. , 2023a ] .”
A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, neutral): “One of the most prominent studies is Reflexion [138], which proposes verbal reinforcement learning for LLM-based agents.”