Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

By Wanjie Sun, Zhenzhong Chen. 2019

166 citations recorded by the source corpus.

Deep convolutional neural network based image super-resolution (SR) models have shown superior performance in recovering the underlying high resolution (HR) images from low resolution (LR) images obtained from the predefined downscaling methods. In this paper, we propose a learned image downscaling method based on content adaptive resampler (CAR) with consideration on the upscaling process. The proposed resampler network generates content adaptive image resampling kernels that are applied to the original HR input to generate pixels on the downscaled image. Moreover, a differentiable upscaling (SR) module is employed to upscale the LR result into its underlying HR counterpart. By back-propagating the reconstruction error down to the original HR input across the entire framework to adjust model parameters, the proposed framework achieves a new state-of-the-art SR performance through upscaling guided image resamplers which adaptively preserve detailed information that is essential to the upscaling. Experimental results indicate that the quality of the generated LR image is comparable to that of the traditional interpolation based method and the significant SR performance gain is achieved by deep SR models trained jointly with the CAR model. The code is publicly available on: https://github.com/sunwj/CAR.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, neutral): “Sa & Sp Image segmentation [34], [129], [146], [148], [150], [154], [156], [216], [217], [218], [219], [220], Image-to-image translation [221], Object detection [110], [111], [147], [148], [164], Semantic image synthesis [222], [223], [224], Computer Image denoising [225], Fine-grained classificatio”
Intermediate Layer Optimization for Inverse Problems using Deep Generative Models (2021, background): “Finally, there is signiﬁcant prior work on deep learning meth-ods that do not rely on pre-trained generators, see e.g. (Lucas et al., 2018; Yu et al., 2019; Liu et al., 2019; Sun & Chen, 2020; Sun et al., 2020; Yang et al., 2019; Tian et al., 2020; Tripathi et al., 2018).”
StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulation (2021, background): “Due to their powerful image synthesis capabilities, they are widely used for various visual tasks including image generation [38], image manipulation [32], super-resolution [27], and domain translation [40].”
Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (2021, methodology): “In general, in image rescaling, the downscaling and upscaling processes are jointly modelled by an encoder-decoder framework [18, 25, 38], so that the downscaling model is optimized for the later upscaling operation.”