Exploiting Diffusion Prior for Real-World Image Super-Resolution

By Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin C. K. Chan, Chen Change Loy. International Journal of Computer Vision, 2023

655 citations recorded by the source corpus.

We present a novel approach to leverage prior knowledge encapsulated in pre-trained text-to-image diffusion models for blind super-resolution. Specifically, by employing our time-aware encoder, we can achieve promising restoration results without altering the pre-trained synthesis model, thereby preserving the generative prior and minimizing training cost. To remedy the loss of fidelity caused by the inherent stochasticity of diffusion models, we employ a controllable feature wrapping module that allows users to balance quality and fidelity by simply adjusting a scalar value during the inference process. Moreover, we develop a progressive aggregation sampling strategy to overcome the fixed-size constraints of pre-trained diffusion models, enabling adaptation to resolutions of any size. A comprehensive evaluation of our method using both synthetic and real-world benchmarks demonstrates its superiority over current state-of-the-art approaches. Code and models are available at https://github.com/IceClear/StableSR.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/315d7a58fada47c5729645f0af8ddfaa0743f82f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Chen Change Loy
2. Luc Van Gool
3. Yu Qiao
4. Lei Zhang
5. Chao Dong
6. Ming-Hsuan Yang
7. Xiaogang Wang
8. Dacheng Tao

Citations in the corpus

• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, neutral): “(2) We employ the two real-world datasets, RealSR [3] and DRealSR [61], by using the same configuration as [55] to center-crop the LR image to 128 × 128 1 .”
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “Another popular way [7, 12, 13, 14] is to use an unconditional pre-trained diffusion model as a prior and modify its reverse path to generate the expected HR image.”
• Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization (2023, methodology): “It is optional but could boost much the Real-ISR performance.”
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “Low-Quality Input Real-ESRGAN+[78] StableSR[75] DiffBIR[49] PASD[85] SUPIR (ours) SUPIR (ours).”
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “StableSR [52] designs a time-aware encoder to control the Stable Diffusion.”