Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

By Fuxiao Liu, Kevin Qinghong Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang. International Conference on Learning Representations, 2023

519 citations recorded by the source corpus.

Despite the promising progress in multi-modal tasks, current large multi-modal models (LMMs) are prone to hallucinating inconsistent descriptions with respect to the associated image and human instructions. This paper addresses this issue by introducing the first large and diverse visual instruction tuning dataset, named Large-scale Robust Visual (LRV)-Instruction. Our dataset comprises 400k visual instructions generated by GPT4, covering 16 vision-and-language tasks with open-ended instructions and answers. Unlike existing studies that primarily focus on positive instruction samples, we design LRV-Instruction to include both positive and negative instructions for more robust visual instruction tuning. Our negative instructions are designed at three semantic levels: (i) Nonexistent Object Manipulation, (ii) Existent Object Manipulation and (iii) Knowledge Manipulation. To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts. GAVIE does not require human-annotated groundtruth answers and can adapt to diverse instruction formats. We conduct comprehensive experiments to investigate the hallucination of LMMs. Our results demonstrate existing LMMs exhibit significant hallucinations when presented with our negative instructions, particularly Existent Object and Knowledge Manipulation instructions. Moreover, we successfully mitigate hallucination by finetuning MiniGPT4 and mPLUG-Owl on LRV-Instruction while improving performance on several public datasets compared to state-of-the-art methods. Additionally, we observed that a balanced ratio of positive and negative instances in the training data leads to a more robust model. Code and data are available at https://github.com/FuxiaoLiu/LRV-Instruction.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c7a7104df3db13737a865ede2be8146990fa4026

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Hallucination of Multimodal Large Language Models: A Survey (2024, unknown): “The definition of hallucination types aligns well with the domain of compositional generalization [79, 121] of VLMs, which investigates visio-linguistic generalization and reasoning abilities.”
A survey on multimodal large language models (2023, unknown): “More specifically, Q-Former [64] only uses 32 learnable tokens to represent an image, which might induce information loss.”
OCRBench: on the hidden mystery of OCR in large multimodal models (2023, unknown): “[31, 32] have developed effective systems to assess LMMs. [15, 33, 22] introduces GPT-4 or human evaluation to assess the output of LMMs. MMBench [34] evaluates LMM using multiple-choice questions across various dimensions of ability.”
Foundation Models Defining a New Era in Vision: A Survey and Outlook (2025, unknown): “…hallucinations using various methods such as VQAs, classiﬁcation of objects etc; 2) construction of ﬁne-tuning data aimed at reducing object hallucinations [184], [187], and algorithmic approaches to minimize hallucinations post-training [188], [189], [190].”
A Survey on Hallucination in Large Vision-Language Models (2024, methodology): “Recent research on “LVLMs as agent” [ Liu et al. , 2023d ] demonstrates impressive fidelity in tasks like OCR-VQA, object tagging, and grounding, validating the viability of this direction.”
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, unknown): “In comparison, our proposed Vision-R1 demonstrates a substantial advantage in reasoning performance, effectively balancing CoT complexity and accuracy.”
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models (2024, background): “Concurrently, a significant body of research is focused on devising and implementing methods to reduce hallucinations in these large models [123, 124, 125].”
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, background): “Despite these advancements, specific challenges persist, with the issue of object hallucination [20, 38, 43, 48] being a prominent concern that impacts the reliability and applicability of LVLMs across domains.”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “Gunjal et al. (2023), Lu et al. (2023), and Liu et al. (2023c) pro-pose to further finetuning the model to produce more truthful and helpful responses.”
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models (2024, methodology): “…al., 2023), obtained by prompting GPT-4V to generate rich synthetic examples from images sourced from LVIS (Gupta et al., 2019), and LRV-Instruct (Liu et al., 2023a) that specifically optimizes for image diversity relative to existing datasets (adding e.g., charts, scientific diagrams, and news…”
Detecting and Preventing Hallucinations in Large Vision Language Models (2023, methodology): “Aligning to Human Preferences Despite having strong zero-shot performance on classical language benchmark datasets, pre-trained LLMs still struggle to produce longer form generations on par with those written by real humans.”
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding (2023, methodology): “In this work, we introduce Chat-UniVi, designed to not only comprehend and generate responses from text but also incorporate visual inputs, thereby providing a more comprehensive and immersive context for response generation.”
A Survey on Evaluation of Large Language Models (2023, background): “[114] introduced a large-scale robust visual instruction dataset to enhance the performance of large-scale multi-modal models in handling relevant images and human instructions.”
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning (2023, background): “The applications of these models extend across diverse domains, including interactive systems, automated content generation, and support for scientific inquiries.”