Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training

By Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, Pascale Fung. 2022

85 citations recorded by the source corpus.

Large-scale vision-language pre-trained (VLP) models are prone to hallucinate non-existent visual objects when generating text based on visual information. In this paper, we systematically study the object hallucination problem from three aspects. First, we examine recent state-of-the-art VLP models, showing that they still hallucinate frequently and models achieving better scores on standard metrics (e.g., CIDEr) could be more unfaithful. Second, we investigate how different types of image encoding in VLP influence hallucination, including region-based, grid-based, and patch-based. Surprisingly, we find that patch-based features perform the best and smaller patch resolution yields a non-trivial reduction in object hallucination. Third, we decouple various VLP objectives and demonstrate that token-level image-text alignment and controlled generation are crucial to reducing hallucination. Based on that, we propose a simple yet effective VLP loss named ObjMLM to further mitigate object hallucination. Results show that it reduces object hallucination by up to 17.4% when tested on two benchmarks (COCO Caption for in-domain and NoCaps for out-of-domain evaluation).

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Pascale Fung
2. Ji-rong Wen
3. Wenliang Dai
4. Dan Su
5. Wayne Xin Zhao
6. Ziwei Ji
7. Zheng Ge
8. Kun Zhou

In Radiology, Nuclear Medicine and Imaging

1. Zheng Ge
2. Jian‐Yuan Sun

In Multimodal Learning

1. Dan Su
2. Ziwei Ji
3. Wenliang Dai
4. Pascale Fung
5. Kun Zhou
6. Ji-rong Wen
7. Tiezheng Yu
8. Jinpeng Wang

In Safety, Robustness & Interpretability

1. Amit P. Sheth
2. Tiezheng Yu
3. Andrea Madotto
4. Nayeon Lee
5. Pascale Fung
6. Saining Xie

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, background): “[28] analyze object hallucination in VL pre-training and propose a novel pre-training objective named object masked language modeling to alleviate this problem.”
• Evaluating Object Hallucination in Large Vision-Language Models (2023, methodology): “They can be formulated as: We select five recently released LVLMs, i.e., mPLUG-Owl (Ye et al., 2023), LLaVA (Liu et al., 2023), Multimodal-GPT (Gong et al., 2023), MiniGPT-4 (Zhu et al., 2023) and Instruct-BLIP (Dai et al., 2023a) and prompt them with following instructions to generate captions abou”
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity (2023, background): “LLMs are known to be susceptible to generating nonfactual, untruthful information, which is referred to as hallucination (Lee et al., 2022; Ji et al., 2022a,b; Su et al., 2022; Dai et al., 2023b; Xu et al., 2023).”
• A Survey on Hallucination in Large Vision-Language Models (2024, background): “More detailed, spatially-aware supervision such as object masked language modeling [Dai et al. , 2023b] for alignment and position-enhanced instruction following [Chen et al. , 2023a] hold immense potentials for further advancements.”
• DreamLLM: Synergistic Multimodal Comprehension and Creation (2023, methodology): “…hallucination, a phenomenon where MLLMs generate non-existent objects or identities in images, significantly compromises their multimodal comprehension capabilities (Dai et al., 2023b; Liu et al., 2023a; Gunjal et al., 2023) and may pose safety risks (MacLeod et al., 2017; Rohrbach et al., 2018).”
• A Survey of Hallucination in Large Foundation Models (2023, background): “Plausible May Not Be Faithful: Probing Object Hallucination in VisionLanguage Pre-training (Dai et al., 2022) Image captioning CHAIR (Rohrbach et al.”