The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution

By Muhammad Waleed Gondal, Bernhard Schölkopf, Michael Hirsch. 2018

52 citations recorded by the source corpus.

While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss [1] alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism and develop a novel semantically guided texture constraining method for further improvement. Using a recently developed perceptual metric employing “deep features” and termed LPIPS [2], the method obtains state-of-the-art results. Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features. Using texture information, off-the-shelf deep classification networks (without training) perform as well as the best performing (tuned and calibrated) LPIPS metrics.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness (2018, methodology): “Furthermore, texture-based generative modelling approaches such as style transfer (Gatys et al., 2016), single image super-resolution (Gondal et al., 2018) as well as static and dynamic texture synthesis (Gatys et al., 2015; Funke et al., 2017) all produce excellent results using standard CNNs,…”
EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning (2019, background): “[24], which has achieved impressive results for style transfer, super-resolution [40, 17], and image-to-image translation [57].”
2018 PIRM Challenge on Perceptual Image Super-resolution (2018, background): “A different approach [8] that achieved high perceptual quality is transferring texture by training with the Gram loss [7], and without adversarial training.”
Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, background): “Gram matrix calculates the feature correlations and usually effectively captures texture information [18].”
The 2018 PIRM Challenge on Perceptual Image Super-Resolution (2018, background): “A different approach [8] that achieved high perceptual quality is transferring texture by training with the Gram loss [7], and without adversarial training.”
EdgeConnect: Structure Guided Image Inpainting using Edge Prediction (2019, background): “[26], which has achieved impressive results for style transfer, super-resolution [42, 18], and image-to-image translation [60].”