DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior

By X. Lin, Jingwen He, Zi-Yuan Chen, Zhaoyang Lyu, Ben Fei, Bo Dai, Wanli Ouyang, Yu Qiao, Chao Dong. arXiv.org, 2023

327 citations recorded by the source corpus.

We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks in a unified framework. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. Each stage is developed independently but they work seamlessly in a cascaded manner. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details. Specifically, IRControlNet is trained based on specially produced condition images without distracting noisy content for stable generation performance. Moreover, we design a region-adaptive restoration guidance that can modify the denoising process during inference without model re-training, allowing users to balance realness and fidelity through a tunable guidance scale. Extensive experiments have demonstrated DiffBIR's superiority over state-of-the-art approaches for blind image super-resolution, blind face restoration and blind image denoising tasks on both synthetic and real-world datasets. The code is available at https://github.com/XPixelGroup/DiffBIR.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/858f0643110ccccb6a9103747f2169fecfb98668

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yu Qiao
2. Chao Dong
3. Luc Van Gool
4. Lei Zhang
5. Chen Change Loy
6. Radu Timofte
7. Xintao Wang
8. Ming-Hsuan Yang

In Media Technology

1. Jinjin Gu
2. Chao Dong
3. Lei Zhang
4. Xintao Wang
5. Yu Qiao
6. Jingwen He
7. Peiran Ren
8. Tao Yang

Citations in the corpus

• Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, unknown): “Given the 16 LR images with the three”
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Low-Quality Input Real-ESRGAN+[78] StableSR[75] DiffBIR[49] PASD[85] SUPIR (ours) Figure 6.”
• Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “The proposed PASD was simple to implement, and our extensive experiments demonstrated its effectiveness and flexibility across different tasks, showing its great potentials for handling complex image restoration and stylization tasks.”
• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, unknown): “(3) We build another real-world dataset, named RealLR200 , which comprises 38 LR images used in recent literature [34, 57, 69], 47 LR images from DiffBIR [40], 50 LR images from Vide-oLQ (the last frame of each video sequence) [4], and 65 LR images collected from the internet by ourselves.”