OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

By Qidong Huang, Xiao-wen Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, Neng H. Yu. Computer Vision and Pattern Recognition, 2023

542 citations recorded by the source corpus.

Hallucination, posed as a pervasive challenge of multi-modal large language models (MLLMs), has significantly impeded their real-world usage that demands precise judgment. Existing methods mitigate this issue with either training with specific designed data or inferencing with external knowledge from other sources, incurring inevitable additional costs. In this paper, we present OPERA, a novel MLLM decoding method grounded in an Over-trust Penalty and a Retrospection-Allocation strategy, serving as a nearly free lunch to alleviate the hallucination issue without additional data, knowledge, or training. Our approach begins with an interesting observation that, most hallucinations are closely tied to the knowledge aggregation patterns manifested in the self-attention matrix, i.e., MLLMs tend to generate new tokens by focusing on a few summary tokens, but not all the previous tokens. Such partial overtrust inclination results in the neglecting of image tokens and describes the image content with hallucination. Based on the observation, OPERA introduces a penalty term on the model logits during the beam-search decoding to mitigate the over-trust issue, along with a rollback strategy that retrospects the presence of summary tokens in the previously generated tokens, and re-allocate the token selection if necessary. With extensive experiments, OPERA shows significant hallucination-mitigating performance on different MLLMs and metrics, proving its effectiveness and generality. Our code is at: https://github.com/shikiw/OPERA.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/49b79d61ffc2db6dce8c2cd9cda06e1876ed8b4c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuhang Cao
2. Yuhang Zang
3. Dahua Lin
4. Xiao-wen Dong
5. Xingcheng Zhang
6. Bin Wang
7. Jiaqi Wang
8. Conghui He

Citations in the corpus

• InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model (2024, background): “…improve the MLLM in different dimensions, such as instruction tuning [11, 49, 98], efficient fine-tuning [33], high-resolution image inputs [6, 82, 83], hallucination mitigation [34, 87, 99], image generation [23, 30, 74, 89], 3D understanding [63] and image-text comprehension and composition [95].”
• ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (2024, unknown): “In Section 2.1, we describe our methods for collecting and filtering video data.”
• Are We on the Right Way for Evaluating Large Vision-Language Models? (2024, unknown): “For example, MiniGPT4 [54] and LLaVA [26] directly connect the vision encoder and LLM with QFormer [22] and MLP [40], showing proficiency in multi-modal dialogues.”
• Hallucination of Multimodal Large Language Models: A Survey (2024, unknown): “Among the various methods, Reinforcement Learning from Human Feedback”
• A Survey on Hallucination in Large Vision-Language Models (2024, background): “A growing body of literature has explored various factors contributing to hallucinations in LLMs [Ji et al. , 2023; Zhang et al. , 2023b; Huang et al. , 2023a].”