Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks

By Yuan Yuan, Siyuan Liu, Jiawei Zhang, Yongbing Zhang, Chao Dong, Liang Lin. 2018

511 citations recorded by the source corpus.

We consider the single image super-resolution problem in a more general case that the low-/high-resolution pairs and the down-sampling process are unavailable. Different from traditional super-resolution formulation, the low-resolution input is further degraded by noises and blurring. This complicated setting makes supervised learning and accurate kernel estimation impossible. To solve this problem, we resort to unsupervised learning without paired data, inspired by the recent successful image-to-image translation applications. With generative adversarial networks (GAN) as the basic component, we propose a Cycle-in-Cycle network structure to tackle the problem within three steps. First, the noisy and blurry input is mapped to a noise-free low-resolution space. Then the intermediate image is up-sampled with a pre-trained deep model. Finally, we fine-tune the two modules in an end-to-end manner to get the high-resolution output. Experiments on NTIRE2018 datasets demonstrate that the proposed unsupervised method achieves comparable results as the state-of-the-art supervised models.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

Citations in the corpus

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks (2018, methodology): “re a deeper network with channel attention [12], achieving the state-of-the-art PSNR performance. Besides supervised learning, other methods like reinforcement learning [26] and unsupervised learning [27] are also introduced to solve general image restoration problems. Several methods have been prop”
Deep Learning for Single Image Super-Resolution: A Brief Review (2018, methodology): “Very recently, by leveraging the extension of the basic GAN framework [122], [123] was proposed as an unsupervised SR algorithm.”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “Thus, implicit modeling attempts to learn a degradation generation process within networks [54, 12, 46].”
CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) (2018, methodology): “Hierarchical features and representations derived from a convolutional neural network (CNN) are leveraged to enhance the discriminative capacity of visual quality, thus people have started developing SR models for natural images [23]–[27].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “Based on CycleGAN [53], the Cyclein-Cycle GANs [65] is proposed for unsupervised image SR.”
Real-World Single Image Super-Resolution: A Brief Review (2021, methodology): “Inspired by CycleGAN [145], the domain translation network in CinCGAN [99] is trained to map realistic LR inputs to synthetic LR images (i.”
Deep Learning on Image Denoising: An overview (2019, methodology): “Unsupervised learning methods [115] use given training samples to find patterns rather than label matching and finish specific tasks, such as unpairing real low-resolution images [250].”
Frequency Separation for Real-World Super-Resolution (2019, background): “[37] propose a model that learns a mapping from the original input to a clean input space, after which they apply super-resolution.”
EnlightenGAN: Deep Light Enhancement Without Paired Supervision (2019, methodology): “…domains by using a cycle-consistent loss with unpaired data A handful of latest works followed their methodology and applied un-paired training with cycle-consistency to several low-level vision tasks, e.g. dehazing, deraining, super-resolution and mobile photo enhancement [31, 32, 33, 34].”
Deep Learning for Image Super-Resolution: A Survey (2019, methodology): “[131] present a cyclein-cycle approach for super-resolution.”
Unsupervised Image Super-Resolution with an Indirect Supervised Path (2019, methodology): “Other researchers [31, 2] make attempt to reduce this gap with unsupervised learning, where cycle-consistency and adversarial losses are utilized to deal with unpaired HR and real LR images.”
Unsupervised Learning for Real-World Super-Resolution (2019, methodology): “Cleaning the input Another strategy for tackling the shift between train and test distribution, caused by the bicubic downsampling, is to map the input image X to the bicubic distribution Ẑ = F (X) ∼ pZ before applying the SR network, as proposed in [37].”
Image-to-Image Translation: Methods and Applications (2021, background): “[22], [23], [24], [25], [26], image super-resolution [27], [28], domain adaptation [29], [30], [31], cartoon generation [32], [33], [34], [35], [36], [37] and image registration [38].”
A Deep Journey into Super-resolution (2019, methodology): “The unsupervised image SR aims to solve this problem by learning a function from unpaired LR-HR image sets [107].”