SRFlow: Learning the Super-Resolution Space with Normalizing Flow

By Andreas Lugmayr, Martin Danelljan, Luc Van Gool, Radu Timofte. 2020

425 citations recorded by the source corpus.

Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image. This fundamental fact is largely ignored by state-of-the-art deep learning based approaches. These methods instead train a deterministic mapping using combinations of reconstruction and adversarial losses. In this work, we therefore propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output given the low-resolution input. Our model is trained in a principled manner using a single loss, namely the negative log-likelihood. SRFlow therefore directly accounts for the ill-posed nature of the problem, and learns to predict diverse photo-realistic high-resolution images. Moreover, we utilize the strong image posterior learned by SRFlow to design flexible image manipulation techniques, capable of enhancing super-resolved images by, e.g., transferring content from other images. We perform extensive experiments on faces, as well as on super-resolution in general. SRFlow outperforms state-of-the-art GAN-based approaches in terms of both PSNR and perceptual quality metrics, while allowing for diversity through the exploration of the space of super-resolved solutions.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

Image Super-Resolution via Iterative Refinement (2021, methodology): “We compare SR3 with EnhanceNet [31], ESRGAN [30], SRFlow [32], FSRGAN [14] and PULSE [17].”
GAN Inversion: A Survey (2021, neutral): “(DBMs) [154], [155], [156], [157], Variational Autoencoders (VAEs) [158], [159], Deep Autoregressive Models (DARs) [160], [161], [162], [163], [164], and Normalizing Flow Models (NFMs) [165], [166], [167], [168].”
Low-Light Image Enhancement with Normalizing Flow (2021, methodology): “(Wolf et al. 2021; Lugmayr et al. 2020; Xiao et al. 2020) The normalizing ﬂow is capable to learn a more complicated conditional distribution than the classical pixel-wise loss, which can well solve the above-mentioned two issues.”
Implicit Diffusion Models for Continuous Super-Resolution (2023, positive): “Deep generative models, including autoregressive [30, 43], GAN-based [15,16,18,26], flow-based [8,24] and variational autoen-coders (VAEs) [17, 42], have emerged as solutions that enrich detailed textures.”
Image Restoration with Mean-Reverting Stochastic Differential Equations (2023, background): “Common specific examples include image deraining (Li et al., 2019; Ren et al., 2019), deblurring (Nah et al., 2017; Zhang et al., 2020), denoising (Zhang et al., 2017a; 2018a), and super-resolution (Dong et al., 2015; Lugmayr et al., 2020; Luo et al., 2022a), just to mention a few.”
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild (2024, background): “Beyond GANs, other generative models can also serve as priors [10, 36, 54, 73, 97–99].”
Stochastic Image-to-Video Synthesis using cINNs (2021, methodology): “[49] Andreas Lugmayr, Martin Danelljan, Luc Van Gool, and”
SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models (2021, positive): “, 2018a], LR-PSNR [Lugmayr et al., 2020] and the pixel standard deviation σ.”
Flow-based Kernel Prior with Application to Blind Super-Resolution (2021, background): “Normalizing flows [8, 9, 22, 24, 28, 29, 32, 37] are invertible generative models that deform the complex data distribution to a simple and tractable distribution.”
SinSR: Diffusion-Based Image Super-Resolution in a Single Step (2023, background): “Subsequently, normalizing flows [23, 42] are demonstrated to have good perceptual quality under an efficient inference process, while its network design is restricted by the requirements of the invertibility and ease of calculation.”
HiNet: Deep Image Hiding by Invertible Network (2021, methodology): “[18] proposed a normalizing flow-based method via INN on superresolution, which attempted to directly account for the illposed nature of super-resolution, and learn to predict diverse photo-realistic high-resolution images.”