DenseCLIP: Extract Free Dense Labels from CLIP

By Chong Zhou, Chen Change Loy, Bo Dai. 2021

82 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

A Simple Framework for Open-Vocabulary Segmentation and Detection (2023, background): “In the light of strong zero-shot transferability demonstrated in the seminal work CLIP [39], a number of researchers have attempted to build advanced open-vocabulary models by leveraging largescale image-text pairs for fine-grained vision tasks like detection [10, 12, 22, 29, 56] and segmentation [7”
GroupViT: Semantic Segmentation Emerges from Text Supervision (2022, background): “The concurrently developed unpublished text-supervised semantic segmentation methods [29, 86, 90, 96] also show promising results.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “Besides these open vocabulary detection settings, recent works [248], [249], [280], [281], [282] perform open vocabulary segmentation.”
ConceptFusion: Open-set Multimodal 3D Mapping (2023, methodology): “To retain knowledge of all concepts captured by the underlying CLIP model, MaskCLIP [27] proposes a zero-shot approach that combines self-attention with engineered 1 × 1 convolution layers for open-set segmentation.”
CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels (2022, methodology): “DenseCLIP (Rao et al. 2022) and MaskCLIP (Zhou, Loy, and Dai 2021) apply it for per-pixel prediction in segmentation.”
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration (2022, methodology): “Zero-shot techniques have become increasingly popular in image classification [53, 33, 51], object detection [4, 54, 19, 38, 77, 42, 26], and semantic segmentation [6, 35, 32, 3, 14, 76].”
Expanding Language-Image Pretrained Models for General Video Recognition (2022, methodology): “For example, CoOp [61], CLIP-Adapter [16] and Tip-Adapter [58] use the pretrained CLIP for improving the few-shot transfer, while PointCLIP [59] and DenseCLIP [38,60] transfer the knowledge to point cloud understanding and dense prediction, respectively.”
Exploring CLIP for Assessing the Look and Feel of Images (2022, methodology): “Through training with massive image-text pairs, CLIP demonstrates exceptional capability in building semantic relationship between texts and visual entities without explicit training [12,14,17,38,42,73].”
Scaling Open-Vocabulary Image Segmentation with Image-Level Labels (2021, methodology): “[62] adapt CLIP for segmentation, and use pseudo per-pixel labels and self-training to boost the performance; similar to Li et al.”