DiffusionDet: Diffusion Model for Object Detection

By Shoufa Chen, Pei Sun, Yibing Song, Ping Luo. IEEE International Conference on Computer Vision, 2022

758 citations recorded by the source corpus.

We propose DiffusionDet, a new framework that formulates object detection as a denoising diffusion process from noisy boxes to object boxes. During the training stage, object boxes diffuse from ground-truth boxes to random distribution, and the model learns to reverse this noising process. In inference, the model refines a set of randomly generated boxes to the output results in a progressive way. Our work possesses an appealing property of flexibility, which enables the dynamic number of boxes and iterative evaluation. The extensive experiments on the standard benchmarks show that DiffusionDet achieves favorable performance compared to previous well-established detectors. For example, DiffusionDet achieves 5.3 AP and 4.8 AP gains when evaluated with more boxes and iteration steps, under a zero-shot transfer setting from COCO to CrowdHuman. Our code is available at https://github.com/ShoufaChen/DiffusionDet.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/81621c53c6aa421deb81bb1359138ded0fb4e258

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Ming-Hsuan Yang
3. Chen Change Loy
4. Dacheng Tao
5. Jie Zhou
6. Ying Shan
7. Jiwen Lu
8. Xiaogang Wang

Citations in the corpus

• A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence (2023, background): “, image segmentation [3, 4, 8, 24, 55, 64], object detection [7], and monocular depth estimation [13, 50].”
• VideoCrafter1: Open Diffusion Models for High-Quality Video Generation (2023, methodology): “With the rapid development of generative models, particularly diffusion models [26, 47], numerous breakthroughs have been achieved in fields such as image generation [11, 12, 16, 19, 39, 41, 43, 57] and video generation [10, 24, 27, 45, 50], as well as in recognition and detection tasks [13, 37].”
• UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models (2023, methodology): “[6] Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo.”
• Diffusion models in medical imaging: A comprehensive survey (2022, methodology): “In the case of object detection, recent vision-based work has demonstrated the potential of applying diffusion models to the object detection task by progressively refining randomly generated boxes to produce the final output results [175].”
• Unleashing Text-to-Image Diffusion Models for Visual Perception (2023, methodology): “Different from previous diffusion-based framework [9, 1] that reformulate the visual perception task as progressive denoising, we employ the denoising autoencoder pre-trained on the text-to-image generation as a backbone and study how to make full use of the learned highlevel and low-level knowledge”