Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model

By Yinhuai Wang, Jiwen Yu, Jian Zhang. International Conference on Learning Representations, 2022

767 citations recorded by the source corpus.

Most existing Image Restoration (IR) models are task-specific, which can not be generalized to different degradation operators. In this work, we propose the Denoising Diffusion Null-Space Model (DDNM), a novel zero-shot framework for arbitrary linear IR problems, including but not limited to image super-resolution, colorization, inpainting, compressed sensing, and deblurring. DDNM only needs a pre-trained off-the-shelf diffusion model as the generative prior, without any extra training or network modifications. By refining only the null-space contents during the reverse diffusion process, we can yield diverse results satisfying both data consistency and realness. We further propose an enhanced and robust version, dubbed DDNM+, to support noisy restoration and improve restoration quality for hard tasks. Our experiments on several IR tasks reveal that DDNM outperforms other state-of-the-art zero-shot IR methods. We also demonstrate that DDNM+ can solve complex real-world applications, e.g., old photo restoration.

Subfields: Biomedical Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3a75ed3e9e81c9db573ef73d20e2c66c12aaedf8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Jian Zhang
2. Yu Qiao
3. Kai Zhang
4. Bo Dai
5. Luc Van Gool
6. Radu Timofte
7. Deyu Meng
8. Tom Goldstein

In Computational Mechanics

1. Radu Timofte
2. Luc Van Gool
3. Chao Dong
4. Kai Zhang
5. Deyu Meng
6. Bernard Ghanem
7. Jian Zhang
8. Yu Qiao

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Radu Timofte
3. Chen Change Loy
4. Ming-Hsuan Yang
5. Lei Zhang
6. Yu Qiao
7. Dacheng Tao
8. Xiaogang Wang

Citations in the corpus

• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion (2023, methodology): “DDNM [64] shares a similar idea with DDRM by refining the null-space of the operator A iteratively for image restoration tasks.”
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior (2023, methodology): “(a) DiffBIR w/o restoration module performs poorly in both fidelity maintaining (first row) and degradation removal (second row); (b) w/o finetuning Stable Diffusion, directly applying the image guidance technique [57; 16] is not able to produce realistic results.”
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, methodology): “IR methods based on generative priors [42, 49, 65, 79] leverage powerful pre-trained generative models to introduce high-quality generation and prior knowledge into IR, bringing significant progress in these aspects.”
• FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model (2023, methodology): “…models (CDMs) with diverse conditions have been proposed, such as text [20, 1, 13, 23, 33, 37, 32, 35, 22, 29], class labels [10], degraded images [5, 6, 7, 8, 19, 24, 39, 41, 44, 34, 36, 45], segmentation maps [28, 49], landmarks [28, 49], hand-drawn sketches [28, 49], style images [28, 49], etc.”
• Universal Guidance for Diffusion Models (2023, methodology): “Prior work that studied guided image generation did so with a variety of restrictions and external guidance functions [3, 4, 6, 7, 12, 16, 28].”
• SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution (2023, unknown): “Some researchers [26, 58] have leveraged pretrained This CVPR paper is the Open Access version, provided by the Computer Vision Foundation.”