Object Hallucination in Image Captioning

By Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, Kate Saenko. 2018

816 citations recorded by the source corpus.

Despite continuously improving performance, contemporary image captioning models are prone to “hallucinating” objects that are not actually in a scene. One problem is that standard metrics only measure similarity to ground truth captions and may not fully capture image relevance. In this work, we propose a new image relevance metric to evaluate current models with veridical visual labels and assess their rate of object hallucination. We analyze how captioning model architectures and learning objectives contribute to object hallucination, explore when hallucination is likely due to image misclassification or language priors, and assess how well current sentence metrics capture object hallucination. We investigate these questions on the standard image captioning benchmark, MSCOCO, using a diverse set of models. Our analysis yields several interesting findings, including that models which score best on standard sentence metrics do not always have lower hallucination and that models which hallucinate more tend to make errors driven by language priors.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic Noise Matters for Neural Natural Language Generation (2019, background): “…work on related tasks, which mostly reports on hallucinations (i.e. adding information not grounded in the input), as observed for image captioning (Rohrbach et al., 2018), sports report generation (Wiseman et al., 2017), machine translation (Koehn and Knowles, 2017; Lee et al., 2019), and…”
Survey of Hallucination in Natural Language Generation (2022, methodology): “Controllability means the ability of models to control the level of hallucination and strike a balance between faithfulness and diversity [41, 159].”
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning (2021, background): “In the second study, we evaluate if using pretrained language models introduces excessive linguistic prior that could cause the known object hallucination problem [62].”
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation (2019, background): “Hallucination (Rohrbach et al., 2018; Lee et al., 2018) refers to when an NLG system generates text which mentions extra information than what is present in the source from which it is generated.”
Connecting Vision and Language with Localized Narratives (2019, background): “Yet, many researchers want deeper visual grounding which links specific words in the caption to specific regions in the image [36,37,49,50].”
Grounded Video Description (2018, background): “While researchers have started to discover and study these problems for image description [14, 9, 24, 20],1 they are even more pronounced”
QuestEval: Summarization Asks for Fact-based Evaluation (2021, background): “…while recent state-of-the-art generative models (Lewis et al., 2019; Zhang et al., 2019a) produce ﬂuent summaries, they frequently contain false or unsupported information (Kry´sci´nski et al., 2019), a phenomenon also known as neural hallucination (Rohrbach et al., 2018; Zhao et al., 2020).”
TRUE: Re-evaluating Factual Consistency Evaluation (2022, background): “…text generation models for real-world applications is that they often generate factually inconsistent text with respect to the input they are conditioned on, or even completely “hallucinate” (Lee et al., 2018; Rohrbach et al., 2018; Maynez et al., 2020; Zhao et al., 2020) as exempliﬁed in Table 1.”
Visual Commonsense R-CNN (2020, methodology): “Moreover, since the existing metrics fall short to the dataset bias, we also applied a new metric CHAIR [55] to measure the object hallucination ( e.g. , “hallucinate” objects not in image).”
Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling (2021, background): “Such grounded captioning ability is important for reducing object hallucination [48], boosting the model’s interpretability and fairness [20,48], and facilitating various robotics and human-computer interaction applications.”
CLIPScore: A Reference-free Evaluation Metric for Image Captioning (2021, background): “Prior work has demonstrated that, for many literal description tasks, humans often prefer correctness in captions over speciﬁcity (Rohrbach et al., 2018, 2017).”
More Grounded Image Captioning by Distilling Image-Text Matching Model (2020, background): “This may lead to undesired behaviors such as object hallucinations [47] and gender discrimination [14], which harm the rationality and explainability of the neural image captioning models.”
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning (2021, neutral): “First and foremost, describing an image while failing to correctly identify objects is not desirable to humans [33].”
Learning to Collocate Neural Modules for Image Captioning (2019, background): “Indeed, image captioning is not the only model that can easily exploit the dataset bias to captioning even without looking at the image [44], almost all existing models for visual reasoning tasks such as visual Q&A [23, 46, 48] have been spotted mode collapse to certain dataset idiosyncrasies and fa”
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2021, background): “Such reference-based hallucination detection has been proposed for ab-stractive summarization (Maynez et al., 2020), machine translation (Wang and Sennrich, 2020), data-to-text generation (Rebuffel et al., 2021), and image caption generation (Rohrbach et al., 2018).”
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models (2023, methodology): “In this section, we investigate the importance and effectiveness of the second-stage finetuning approach.”
Mitigating Gender Bias in Captioning Systems (2020, methodology): “From the model architecture perspective, we consider models both with and without attention module where "attention" refers to models learn to pay attention to different image regions for caption word generation [30].”
Analyzing and Mitigating Object Hallucination in Large Vision-Language Models (2023, methodology): “…evaluate LURE on multiple existing open-source LVLMs. Compared to the best prior method, the results demonstrate that LURE can significantly reduce object hallucination under general object hallucination evaluation metrics (e.g., CHAIR (Rohrbach et al., 2018)), GPT evaluation, and human evaluation.”