Open-vocabulary Attribute Detection
By María Alejandra Bravo, Sudhanshu Mittal, Simon Ging, Thomas Brox. 2022
42 citations recorded by the source corpus.
Vision-language modeling has enabled open-vocabulary tasks where predictions can be queried using any text prompt in a zero-shot manner. Existing open-vocabulary tasks focus on object classes, whereas research on object attributes is limited due to the lack of a reliable attribute-focused evaluation benchmark. This paper introduces the Open- Vocabulary Attribute Detection (OVAD) task and the corresponding OVAD benchmark. The objective of the novel task and benchmark is to probe object-level attribute information learned by vision-language models. To this end, we created a clean and densely annotated test set covering 117 attribute classes on the 80 object classes of MS COCO. It includes positive and negative annotations, which enables open-vocabulary evaluation. Overall, the benchmark consists of 1.4 million annotations. For reference, we provide a first baseline method for open-vocabulary attribute detection. Moreover, we demonstrate the benchmark's value by studying the attribute detection performance of several foundation models.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- D'idac Sur'is
- Sachit Menon
- Carl Vondrick
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Cordelia Schmid
- Andrew Zisserman
Citations in the corpus
- ViperGPT: Visual Inference via Python Execution for Reasoning (2023, methodology): “The perception and external knowledge modules used by ViperGPT are GLIP [31] for object detection, X-VLM [64] for text-image similarity (as it surpasses CLIP [43] at attribute detection [5]), MiDaS [44] for depth estimation, GPT-3 [6] for external knowledge, and BLIP-2 [30] for simple visual queries”