Evaluating Object Hallucination in Large Vision-Language Models

By Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, Ji-rong Wen. 2023

1,825 citations recorded by the source corpus.

Inspired by the superior language abilities of large language models (LLM), large vision-language models (LVLM) have been recently explored by integrating powerful LLMs for improving the performance on complex multimodal tasks. Despite the promising progress on LVLMs, we find that LVLMs suffer from the hallucination problem, i.e. they tend to generate objects that are inconsistent with the target images in the descriptions. To investigate it, this work presents the first systematic study on object hallucination of LVLMs. We conduct the evaluation experiments on several representative LVLMs, and show that they mostly suffer from severe object hallucination issue. We further discuss that the visual instructions may influence the hallucination, and find that: objects that frequently occur in the visual instructions or co-occur with the image objects, are obviously prone to be hallucinated by LVLMs. Besides, we find that existing evaluation methods might be affected by the input instructions and generation styles of LVLMs. Thus, we further design an improved evaluation method for object hallucination by proposing a polling-based query method called POPE. Experiment results demonstrate that our POPE can evaluate the object hallucination in a more stable and flexible way. Our codes and data are publicly available at https://github.com/RUCAIBox/POPE.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Multimodal Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Multimodal Learning

In Safety, Robustness & Interpretability

Citations in the corpus

Improved Baselines with Visual Instruction Tuning (2023, methodology): “For recent benchmarks proposed for instruction-following LMMs, POPE [33] evaluates model’s degree of hallucination on three sampled subsets of COCO [34]: random, common, and adversarial and we report the F1 score on all three splits.”
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs (2024, methodology): “We also evaluate on POPE [27] which is designed to test hallucination in visual grounding.”
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation (2024, positive): “In SEED-X, we adopt a visual tokenizer to unify image comprehension and generation, and pre-train a multi-granularity de-tokenizer to facilitate image generation and high-precision image manipulation in a two-stage manner.”
CogAgent: A Visual Language Model for GUI Agents (2023, positive): “Furthermore, we conducted zero-shot tests of our model on the challenging MM-Vet [41] and POPE [19] datasets, both of which are instrumental in gauging the multi-modal capabilities and the generalization performance in complex tasks including conversation question-answering, detailed descriptions,…”
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, positive): “POPE [139]: POPE is a benchmark for evaluating object hallucination in MLLMs, utilizing binary questions to quantify and analyze hallucination tendencies.”
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding (2023, background): “Despite this, the generated description from VideoChat includes several hallucinations [21], such as references to “pepper” and “avocados” that are not present in the video.”
Detecting and Preventing Hallucinations in Large Vision Language Models (2023, background): “Hallucination Analysis in LVLMs Recent research [11] undertakes a systematic analysis of object hallucinations in large vision-language models (LVLMs).”
DeepSeek-VL: Towards Real-World Vision-Language Understanding (2024, methodology): “As illustrated in Figure 7, DeepSeek-VL is judged to be superior in over 60% of instances when compared to open-source multimodal models, including Fuyu-8B, CogVLM-17B, and InternLM-XComposer2-VL.”
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action (2023, background): “Vision-language results on nine tasks [1, 28, 59, 91, 124, 129, 130, 136, 209] and three evaluation-only benchmarks [106, 113, 120].”
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model (2024, neutral): “We report results in Math-Vista [52], MMMU [91], AI2D [40], MME [27], MM-Bench [51], MMBench-Chinese [51], SEED-Bench (Image) [41], LLaVA-Bench (In-the-Wild) [49], QBench [85], MM-Vet [90], HallusionBench [31], ChartQA [56], and POPE [45].”
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation (2024, positive): “To assess multimodal understanding capabilities, we evaluate our model on widely recognized image-based vision-language benchmarks, which include VQAv2 [30], GQA [33], POPE [46], MME [24], SEED [40], MMB [52], MM-Vet [87], and MMMU [88].”
SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models (2023, methodology): “We test our model on recently proposed MLLM benchmarks to comprehensively evaluation of the model’s characteristic such as MME (Fu et al., 2023b), Seedbench (Li et al., 2023c), POPE (Li et al., 2023e), LLaVA-Bench (In-the-Wild) (Liu et al., 2023d), MM-Vet (Yu et al., 2023b), MathVista (Lu et al., 20”
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models (2024, positive): “We evaluate SPHINX-X on recently introduced benchmarks, such as MME (Fu et al., 2023a), Seedbench (Li et al., 2023a), POPE (Li et al., 2023f), LLaVA-Bench (In-the-Wild) (Liu et al., 2023b), MM-Vet (Yu et al., 2023b), MathVista (Lu et al., 2023), MM-bench (Liu et al., 2023d), CCbench (Contributors,…”
A survey on multimodal large language models (2023, positive): “[44] delve into the object hallucination problem and propose a query method POPE to assess performance in this regard.”
LVLM-EHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models (2023, positive): “To this end, we evaluate the object hallucination problem of LVLMs on the MSCOCO dataset [24] using the POPE pipeline [15].”
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models (2023, methodology): “As for image-based evaluation, we conduct experiments on several widely-adopted benchmarks, including GQA [22], MMB (MMBench) [36], MME [18], POPE [31], SEED [28], SQA I (Image-based setting in ScienceQA) [37], VQA T (TextVQA) [50], VizWiz [20], and VQA v2 (VQA V2) [19].”