Survey of Hallucination in Natural Language Generation
By Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Delong Chen, Wenliang Dai and 2 more. 2022
5,175 citations recorded by the source corpus.
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation, and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions, and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Quoc V. Le
- Graham Neubig
- Yejin Choi
- Yoshua Bengio
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Ming-Hsuan Yang
- Andrew Zisserman
- Jitendra Malik
- Cordelia Schmid
In Natural Language Processing
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Graham Neubig
- Yejin Choi
- Percy Liang
- Caiming Xiong
- Richard Socher
In Safety, Robustness & Interpretability
- Kai-Wei Chang
- Percy Liang
- Nanyun Peng
- Sunipa Dev
- Dan Hendrycks
- Sebastian Gehrmann
- Adina Williams
- Frederick Liu
Citations in the corpus
- Toolformer: Language Models Can Teach Themselves to Use Tools (2023, background): “…to access up-to-date information on recent events (Komeili et al., 2022) and the related tendency to hallucinate facts (Maynez et al., 2020; Ji et al., 2022), difficulties in understanding low-resource languages (Lin et al., 2021), a lack of mathematical skills to perform precise…”
- Autoregressive Search Engines: Generating Substrings as Document Identifiers (2022, background): “…for query expansion in traditional search engines (Mao et al., 2021), but these solutions don’t exploit the full potential of autoregressive architecture, such as word order sensitivity and conditional probability modeling, and still lag behind vector-based approaches (Karpukhin et al., 2020).”
- Multimodal Chain-of-Thought Reasoning in Language Models (2023, neutral): “To elicit CoT reasoning in multimodality, we advocate a Multimodal-1”
- Accurate structure prediction of biomolecular interactions with AlphaFold 3 (2024, background): “The biggest issue is that generative models are prone to hallucination 35 , whereby the model may invent plausible-looking structure even in unstructured regions.”
- PaLM 2 Technical Report (2023, background): “While hallucinations have been described extensively in work on factuality and grounding (Ji et al., 2023), this finding highlights the potential for additional representational harm as well.”
- Teaching Small Language Models to Reason (2022, background): “The concerns here are that both the teacher and student model may potentially generate non-factual (Ji et al., 2022; Pagnoni et al., 2021; Kreps et al., 2022) or offensive output (Gehman et al., 2020).”
- Generative artificial intelligence (2023, methodology): “…variance property of GAI, generative models are not immune to output errors, or so-called hallucinations , which manifest themselves in confidently generated results that seem plausible but are unreasonable with respect to the source of information (Ji et al., 2023; Susarla et al., 2023).”
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023, background): “generations across various applications (Ji et al., 2023; Bang et al., 2023; Zhao et al., 2023), where the generated content is either conflict with existing source or cannot be verified by the available knowledge resources.”
- Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2022, background): “Despite the success of deep learning models, they suffer the hallucination problem, which degrades the performance and hinders practical applications (Ji et al., 2022).”
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023, background): “Hallucination has been studied in text generation tasks, including summarization (Huang et al., 2021) and dialogue generation (Shuster et al., 2021), as well as in a variety of other natural language generation tasks (Ji et al., 2023).”
- Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, background): “Prior to the advent of LLMs, the NLP community has primarily defined “hallucination” as the generation of non-sensical content or content that deviates from its sources [28, 32, 39, 57, 74, 76].”
- INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2024, background): “Recent works have shown that the LLMs are subject to the risks of self-consistent (overconfident) hallucinations (Ren et al., 2022; Ji et al., 2023), which has not been considered by existing consistency based methods.”
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors (2022, neutral): “For example, Large Language Models (LLMs) notoriously hallucinate [17], perpetuate bias [11], and factually decay [8].”
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023, methodology): “Hallucinations in LLMs. Hallucinations in LLMs refer to generated content not based on training data or facts, caused by various factors like imperfect learning and decoding (Ji et al., 2023).”
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (2023, background): “However, LLMs are known for having hallucination issues, which are prone to generating incorrect plans and leading to wrong answers (Ji et al., 2023).”
- Knowledge Graph Prompting for Multi-Document Question Answering (2023, methodology): “To mitigate the hallucination issue and enhance the reasoning capability (Wei et al. 2022; Ji et al. 2023) of the LLM traversal agent, we further instruction fine-tune f (Chung et al. 2022) by predicting the next supporting facts based on previous supporting facts, thereby integrating commonsense…”