Recognize Anything: A Strong Image Tagging Model

By Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu and 2 more. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2023

432 citations recorded by the source corpus.

We present the Recognize Anything Model (RAM): a strong foundation model for image tagging. RAM makes a substantial step for foundation models in computer vision, demonstrating the zero-shot ability to recognize any common category with high accuracy. By leveraging large-scale image-text pairs for training instead of manual annotations, RAM introduces a new paradigm for image tagging.The development of RAM comprises four key steps. Firstly, annotation-free image tags are obtained at scale through automatic text semantic parsing. Subsequently, a preliminary model is trained for automatic annotation by unifying the captioning and tagging tasks, supervised by the original texts and parsed tags, respectively. Thirdly, a data engine is employed to generate additional annotations and clean incorrect ones. Lastly, the model is retrained with the processed data and fine-tuned using a smaller but higher-quality dataset.We evaluate the tagging capability of RAM on numerous benchmarks and observe an impressive zero-shot performance, which significantly outperforms CLIP and BLIP. Remarkably, RAM even surpasses fully supervised models and exhibits a competitive performance compared with the Google tagging API. We have released RAM at https://recognize-anything.github.io/ to foster the advancement of foundation models in computer vision.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9541cf136f442e992f10021c53081f33c73a2ed0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model (2024, methodology): “To this end, we first employing an open-vocabulary image tagging model [34] to identify all the object classes present in the image.”
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, positive): “In specific, we use the methods in [16] , [28] and [77] to extract classification-style, caption-style and tag-style prompts, respectively.”
GLaMM: Pixel Grounding Large Multimodal Model (2023, methodology): “For Level-1, we use Tag2Text [14] and RAM [58] for image tagging, Co-DETR [62], EVAv02 [7], OWL-ViT [33], and POMP [40] for object localization, GRiT [48] and GPT4RoI [57] for attribute generation, and MiDAS [39] for depth estimation.”
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (2024, methodology): “In the field of computer vision, significant advancements have been made across a variety of tasks, including image recognition [47, 31, 18, 83, 17], generic object detection [49, 87, 43, 27, 77, 36, 19, 38, 51, 50, 20, 30], generic image segmentation [9, 8, 26, 29, 78, 88, 79, 25, 79, 16, 28],…”
Step1X-Edit: A Practical Framework for General Image Editing (2025, neutral): “5-VL [3 ] and the Recognize-Anything Model [69] to identify target objects or keywords, followed by Flux-Fill [6] for content-aware inpainting.”
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (2024, methodology): “Therefore, we made an initial attempt by carefully assigning semantic segmentation labels to our unlabeled images with a combination of RAM [85] + GroundingDINO [37] + HQ-SAM [26] models.”
A Survey on Hallucination in Large Vision-Language Models (2024, background): “A growing body of literature has explored various factors contributing to hallucinations in LLMs [Ji et al. , 2023; Zhang et al. , 2023b; Huang et al. , 2023a].”
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning (2023, methodology): “We also develop a variant of our system, ConceptGraphs - Detector ( CG-D ), where we employ an image tagging model (RAM [55]) to list the object classes present in the image and an open-vocabulary 2D detector (Grounding DINO [34]) to obtain object bounding boxes 6 .”
OpenEQA: Embodied Question Answering in the Era of Foundation Models (2024, neutral): “We use the publicly released implementation of CG, which uses Grounded-SAM [18, 28] with RAM [52] for object detection and LLaVA-v1 [27] for image captioning.”