Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs

By Yu-Sheng Chen, Man-Hsin Kao, Yung-Yu Chuang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

466 citations recorded by the source corpus.

This paper proposes an unpaired learning method for image enhancement. Given a set of photographs with the desired characteristics, the proposed method learns a photo enhancer which transforms an input image into an enhanced image with those characteristics. The method is based on the framework of two-way generative adversarial networks (GANs) with several improvements. First, we augment the U-Net with global features and show that it is more effective. The global U-Net acts as the generator in our GAN model. Second, we improve Wasserstein GAN (WGAN) with an adaptive weighting scheme. With this scheme, training converges faster and better, and is less sensitive to parameters than WGAN-GP. Finally, we propose to use individual batch normalization layers for generators in two-way GANs. It helps generators better adapt to their own input distributions. All together, they significantly improve the stability of GAN training for our application. Both quantitative and visual results show that the proposed method is effective for enhancing images.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/34d6e3da80ee25840f6dadf36b27717da0bb609b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Distilling portable Generative Adversarial Networks for Image Translation (2020, unknown): “…applied to a number of image-to-image translation tasks such as image synthesis (Karras et al. 2017), domain translation (Zhu et al. 2017; Isola et al. 2017; Choi et al. 2018; Huang et al. 2018; Lee et al. 2018), image denoising (Chen et al. 2018a) and image super-resolution (Ledig et al. 2017).”
Image-to-Image Translation: Methods and Applications (2021, background): “Image quality improvement [2], [207], [208], [210], [209] focuses on producing noticeably fewer colored artifacts around hard edges and more accurate colors, as well as reduced noise in smooth shadows.”
Learning Image-Adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-Time (2020, methodology): “The ﬁrst category of methods learn a dense pixel-to-pixel mapping between the input and output image pairs [7], or predict pixel-wise transformations to enhance the input image [2], [6], [10].”
Learning to Restore Low-Light Images via Decomposition-and-Enhancement (2020, background): “Typical image enhancement methods [46, 51, 24, 7, 40, 34, 48, 4] propose to recover low-light images to match with human perception.”
Paint by Example: Exemplar-based Image Editing with Diffusion Models (2022, background): “Deep neural networks can now produce compelling results for various low-level image editing tasks, such as image inpainting [61, 68], composition [42, 67, 72], colorization [54,71,74] and aesthetic enhancement [8,12], by learning from richly available paired data.”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, unknown): “Notable works employ Retinex-inspired networks [4], [86], [87], [88], encoder-decoder networks [89], [90], [91], [92], [93], and GANs [94], [95], [96].”
Underexposed Photo Enhancement Using Deep Illumination Estimation (2019, methodology): “We compare our method with the following five state-ofthe-art image enhancement methods: (i) the latest Retinexbased method, JieP [4], and (ii)-(v) four recent deeplearning-based methods of HDRNet [13], DPE [9], White- Box [15], and Distort-and-Recover [22].”
PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN With Dual-Discriminators (2023, methodology): “Unlike the existing GAN-based UIE methods [27], [28], [29], our Dual-D includes two discrimina-”
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation (2021, methodology): “CNN-based models usually use self-captured paired data for network training [7], [25] while GAN-based models elaborately select unpaired data [9], [37].”
Beyond Brightening Low-light Images (2021, methodology): “In addition, the above methods do not explicitly cope with the degradation of color Most existing methods manipulate the illumination by 1) gamma correction, 2) appointing a level existing in carefully constructed training data, or 3) fusion of different illumination maps.”
MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “Input Target CycleGAN [121] Exposure [31] DPE [18] EnlightenGAN UEGAN [63] MAXIM-2S”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, methodology): “Method HDRNet [36] W-Box [45] DR [75] DPE [19] DeepUPE [100] MIRNet (Ours)”
An Experiment-Based Review of Low-Light Image Enhancement Methods (2020, background): “In contrast to supervised learning methods, generative adversarial network (GAN)-based methods can be used for image enhancement without training on pairs of images [276], [277].”
Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2021, methodology): “Moreover, although some general photo enhancement methods [45], [46], [47], [48], [49], [50], [51], [52], [53] can improve the brightness of images to some extent, we omit them in this survey as they are not designed to handle diverse low-light conditions.”
SNR-Aware Low-light Image Enhancement (2022, unknown): “Also, there are unsupervised methods [7, 14, 19].”
Fast Underwater Image Enhancement for Improved Visual Perception (2019, methodology): “Such models have been effectively used for unpaired learning of perceptual image enhancement [15] as well.”
EnlightenGAN: Deep Light Enhancement Without Paired Supervision (2019, background): “dehazing, deraining, super-resolution and mobile photo enhancement [31, 32, 33, 34].”
Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification (2019, methodology): “Recently developed GANs provide a powerful tool for image translation [29, 12, 1].”