Hallucination In Object Detection — A Study In Visual Part VERIFICATION

By Osman Semih Kayhan, Bart Vredebregt, Jan van Gemert. 2021

20 citations recorded by the source corpus.

We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes 1, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.1https://github.com/oskyhn/DelftBikes

Subfields: Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Pascale Fung
2. Wenliang Dai
3. Ziwei Ji
4. Dan Su
5. Tiezheng Yu
6. Yejin Bang
7. Andrea Madotto
8. Etsuko Ishii

In Safety, Robustness & Interpretability

1. Tiezheng Yu
2. Andrea Madotto
3. Nayeon Lee
4. Pascale Fung

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, background): “Nevertheless, recent works have started to refer to a speciic type of error as "hallucination" in image captioning [113] and object detection [62], which denotes non-existing objects detected or localized incorrectly at their expected position.”