LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples
By Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Munan Ning, Li Yuan. arXiv.org, 2023
319 citations recorded by the source corpus.
Large Language Models (LLMs), including GPT-3.5, LLaMA, and PaLM, seem to be knowledgeable and able to adapt to many tasks. However, we still cannot completely trust their answers, since LLMs suffer from \textbf{hallucination}\textemdash fabricating non-existent facts, deceiving users with or without their awareness. However, the reasons for their existence and pervasiveness remain unclear. In this paper, we demonstrate that nonsensical prompts composed of random tokens can also elicit the LLMs to respond with hallucinations. Moreover, we provide both theoretical and experimental evidence that transformers can be manipulated to produce specific pre-define tokens by perturbing its input sequence. This phenomenon forces us to revisit that \emph{hallucination may be another view of adversarial examples}, and it shares similar characteristics with conventional adversarial examples as a basic property of LLMs. Therefore, we formalize an automatic hallucination triggering method as the \textit{hallucination attack} in an adversarial way. Finally, we explore the basic properties of attacked adversarial prompts and propose a simple yet effective defense strategy. Our code is released on GitHub\footnote{https://github.com/PKU-YuanGroup/Hallucination-Attack}.
Subfields: Artificial Intelligence, Health Informatics
View Paper: https://www.semanticscholar.org/paper/dd4d82299b4209db539d639f836fcee663cf72b3
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- ChatGPT: The End of Online Exam Integrity? (2022, unknown): “Hallucination in LLMs should not come as a surprise—LLMs hallucinate by design [28,46].”
- Retrieval-Augmented Generation for Large Language Models: A Survey (2023, methodology): “Therefore, this paper aims to comprehensively summarize and organize the technical principles, developmental history, content, and, in particular, the relevant methods and applications after the emergence of LLMs, as well as the evaluation methods and application scenarios of RAG.”
- Evaluation of Retrieval-Augmented Generation: A Survey (2024, background): “…fusion of retrieval and generation capabilities enables the creation of responses that are not only contextually appropriate but also informed by the most current and accurate information available, making RAG a development in the pursuit of more intelligent and versatile language models [56,52].”
- A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly (2023, methodology): “Similarly, Libro [131], a framework that uses LLMs to automatically generate test cases to reproduce software security bugs.”
- When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2025, unknown): “…frequently contain hallucinations or factual inconsistencies that compromise verification reliability (Wang et al., 2024c), and the generated rationales often lack direct grounding in verifiable evidence sources (Huang et al., 2023; Su et al., 2023; Yao et al., 2023; Chen et al., 2024).”