A Survey on Hallucination in Large Vision-Language Models
By Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rong-Zhi Li, Wei Peng. arXiv.org, 2024
376 citations recorded by the source corpus.
Recent development of Large Vision-Language Models (LVLMs) has attracted growing attention within the AI landscape for its practical implementation potential. However, ``hallucination'', or more specifically, the misalignment between factual visual content and corresponding textual generation, poses a significant challenge of utilizing LVLMs. In this comprehensive survey, we dissect LVLM-related hallucinations in an attempt to establish an overview and facilitate future mitigation. Our scrutiny starts with a clarification of the concept of hallucinations in LVLMs, presenting a variety of hallucination symptoms and highlighting the unique challenges inherent in LVLM hallucinations. Subsequently, we outline the benchmarks and methodologies tailored specifically for evaluating hallucinations unique to LVLMs. Additionally, we delve into an investigation of the root causes of these hallucinations, encompassing insights from the training data and model components. We also critically review existing methods for mitigating hallucinations. The open questions and future directions pertaining to hallucinations within LVLMs are discussed to conclude this survey.
Subfields: Cognitive Neuroscience, Neurology
View Paper: https://www.semanticscholar.org/paper/fc4c380102d6f72657d1ab54dffd6be536bb01c7
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
In Neurology
Citations in the corpus
- Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models (2024, background): “Due to the large scale of the training dataset and training methodology of large foundation models (e.g., ChatGPT [100] and Sora [3]), the truthfulness of these models needs to be enhanced as the related issues like hallucination have been discussed widely [119].”
- Large language models for generative information extraction: a survey (2023, neutral): “These challenges include the misalignment between natural language output and structured form [6], hallucination problem in LLMs [48], contextual dependence, high computational resource requirements [49], di ffi culties in updating internal knowledge [50], etc.”