Deep Generative Adversarial Compression Artifact Removal

By Leonardo Galteri, Lorenzo Seidenari, Marco Bertini, A. del Bimbo. 2017

229 citations recorded by the source corpus.

Compression artifacts arise in images whenever a lossy compression algorithm is applied. These artifacts eliminate details present in the original image, or add noise and small structures; because of these effects they make images less pleasant for the human eye, and may also lead to decreased performance of computer vision algorithms such as object detectors. To eliminate such artifacts, when decompressing an image, it is required to recover the original image from a disturbed version. To this end, we present a feed-forward fully convolutional residual network model trained using a generative adversarial framework. To provide a baseline, we show that our model can be also trained optimizing the Structural Similarity (SSIM), which is a better loss with respect to the simpler Mean Squared Error (MSE). Our GAN is able to produce images with more photorealistic details than MSE or SSIM based networks. Moreover we show that our approach can be used as a pre-processing step for object detection in case images are degraded by compression to a point that state-of-the art detectors fail. In this task, our GAN method obtains better performance than MSE or SSIM trained networks.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

High-Fidelity Generative Image Compression (2020, background): “Beyond (un)conditional generation, adversarial losses led to advances in different image enhancement tasks, such as compression artifact removal [15], image de-noising [11], and image super-resolution [25].”
Palette: Image-to-Image Diffusion Models (2021, methodology): “[Dong et al. 2015] applied deep CNN architectures for JPEG restoration, and [Galteri et al. 2017, 2019] successfully applied GANs for artifact removal, but they have been restricted to quality factors above 10.”
Generative Adversarial Networks for Extreme Learned Image Compression (2018, background): “Other works use adversarial training for compression artifact removal (for engineered codecs) (Galteri et al., 2017) and single image super-resolution (Ledig et al., 2017).”
Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, unknown): “To achieve visually-pleasing results, generative adversarial network [19] is usually employed as loss supervisions to push the solutions closer to the natural manifold [43, 59, 66, 7, 15], while our work attempts to leverage the pretrained GANs as generative facial priors (GFP).”
Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement (2019, background): “Fueled by the high performance of deep learning, several works introduce it to enhance the quality of compressed videos and get promising results, including ARCNN [5], deep residual denoising neural networks (DnCNN) [39], generative adversarial networks [6] and multi-frame quality enhancement networ”
Blind Face Restoration via Deep Multi-scale Component Dictionaries (2020, background): “, image super-resolution [9,19,24,44, 46], denoising [13, 38, 42, 43], deblurring [22, 29, 41], and compression artifact removal [8, 10, 12].”
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff (2019, methodology): “In recent years, lossy compression is seeing a surge of research, due in part to the advancements in deep learning and their application in this domain (Toderici et al., 2016; 2017; Ballé et al., 2016; 2017; 2018; Agustsson et al., 2017; 2018; Rippel & Bourdev, 2017; Minnen et al., 2018; Li et al., ”
Deep Generative Models for Distribution-Preserving Lossy Compression (2018, methodology): “GANs were used in the context of learned image compression before [4, 15, 34, 35].”
Deep Network Interpolation for Continuous Imagery Effect Transition (2018, background): “CNN-based approaches have led to a series of breakthroughs for several image restoration tasks including super-resolution [3, 18, 25, 19, 33, 20, 43], denoising [1, 44], de-blocking [40, 7] and deblurring [42, 32, 28].”