DIRE for Diffusion-Generated Image Detection
By Zhendong Wang, Jianmin Bao, Wen-gang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, Houqiang Li. IEEE International Conference on Computer Vision, 2023
554 citations recorded by the source corpus.
Diffusion models have shown remarkable success in visual synthesis, but have also raised concerns about potential abuse for malicious purposes. In this paper, we seek to build a detector for telling apart real images from diffusion-generated images. We find that existing detectors struggle to detect images generated by diffusion models, even if we include generated images from a specific diffusion model in their training data. To address this issue, we propose a novel image representation called DIffusion Reconstruction Error (DIRE), which measures the error between an input image and its reconstruction counterpart by a pre-trained diffusion model. We observe that diffusion-generated images can be approximately reconstructed by a diffusion model while real images cannot. It provides a hint that DIRE can serve as a bridge to distinguish generated and real images. DIRE provides an effective way to detect images generated by most diffusion models, and it is general for detecting generated images from unseen diffusion models and robust to various perturbations. Furthermore, we establish a comprehensive diffusion-generated benchmark including images generated by various diffusion models to evaluate the performance of diffusion-generated image detectors. Extensive experiments on our collected benchmark demonstrate that DIRE exhibits superiority over previous generated-image detectors. The code, models, and dataset are available at https://github.com/ZhendongWang6/DIRE.
Subfields: Biophysics, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/d9d82eb6a8886226724ea230a7e5923d660a0bad
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biophysics
- Richard Zhang
- Eli Shechtman
- Taesung Park
- William T. Freeman
- Michael Gharbi
- Noah A. Smith
- Ranjay Krishna
- Frédo Durand
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
- Li Fei-Fei
Citations in the corpus
- GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image (2023, methodology): “DIRE [33] computes an error between the input and its reconstruction counterpart by a diffusion model.”
- Improved Distribution Matching Distillation for Fast Image Synthesis (2024, background): “To mitigate these risks, we are interested in developing monitoring mechanisms to detect and prevent misuse [74,75] and methods to enhance output diversity and fairness [76].”
- BLINK: Multimodal Large Language Models Can See but Not Perceive (2024, methodology): “Surprisingly, we find that the Caption + LLM setting achieves better [70] DepthAnything [83] LoFTR [68] DIFT [70] DIRE [79] Ordinal Shading [14] 96.51 Effect of visual prompting on Blink : Several Blink tasks involve visual prompting.”