EnlightenGAN: Deep Light Enhancement Without Paired Supervision

By Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, Zhangyang Wang. 2019

2,438 citations recorded by the source corpus.

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data? As one such example, this paper explores the low-light image enhancement problem, where in practice it is extremely challenging to simultaneously take a low-light and a normal-light photo of the same visual scene. We propose a highly effective unsupervised generative adversarial network, dubbed EnlightenGAN, that can be trained without low/normal-light image pairs, yet proves to generalize very well on various real-world test images. Instead of supervising the learning using ground truth data, we propose to regularize the unpaired training using the information extracted from the input itself, and benchmark a series of innovations for the low-light image enhancement problem, including a global-local discriminator structure, a self-regularized perceptual loss fusion, and the attention mechanism. Through extensive experiments, our proposed approach outperforms recent methods under a variety of metrics in terms of visual quality and subjective user study. Thanks to the great flexibility brought by unpaired training, EnlightenGAN is demonstrated to be easily adaptable to enhancing real-world images from various domains. Our codes and pre-trained models are available at: https://github.com/VITA-Group/EnlightenGAN.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “These encoder-decoder GANs also gain popularity in image enhancement (Wang et al., 2018a; Kupyn et al., 2019; Jiang et al., 2019) and editing (Sanakoyeu et al., 2018; Yang et al., 2019).”
AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “It has achieved great success in many generation and synthesis tasks, such as text-to-image translation [66, 65, 61, 48], image-to-image translation [22, 70, 63], and image enhancement [33, 31, 23].”
URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement (2022, background): “Due to these limitations existing in model-based methods, researchers take advantage of deep networks [3,11,15, 25,34,42] to restore low-light images in a data-driven manner.”
Artificial intelligence in the creative industries: a review (2020, background): “Contrast enhancement, along with other methods to be discussed later, suffer from a fundamental lack of data for supervised training because real image pairs with low and high contrast are unavailable (Jiang et al. 2021).”
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement (2023, methodology): “With the rapid progress of deep learning, CNN [15, 16, 21, 22, 25, 34, 36, 38, 45, 48, 60, 65, 67] has been widely used in low-light image enhancement.”
Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement (2020, methodology): “In addition, dozens of methods also train their networks without paired supervision [40, 12, 8].”
Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study (2019, background): “[111] proposed for the first time an unsupervised generative adversarial network, that can be trained without low/normal-light image pairs, yet generalizing nicely and flexibly on various real-world images.”
PSD: Principled Synthetic-to-Real Dehazing Guided by Physical Priors (2021, methodology): “Following the protocol set by EnlightenGAN [13], we ask ten subjects to independently compare the results of the three methods in a pairwise manner.”
Low-Light Image Enhancement With Semi-Decoupled Decomposition (2020, background): “To address this issue, Jiang et al. [24] proposed a highly effective unsupervised generative adversarial network to solve this problem.”
Learning a Simple Low-Light Image Enhancer from Paired Low-Light Instances (2023, methodology): “…be divided into the following three categories: traditional methods (SDD [9], STAR [34]), supervised approaches (MBLLEN [20], RetinexNet [32], GALD-Net [30], KinD [38], DRBN [36], URetinexNet [33]), and unsupervised methods (Zero-DCE [7], RRDNet [41], RUAS [18], SCI [22], and EnlightenGAN [12]).”
Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method (2022, methodology): “Recently, many learning-based LLIE methods have been introduced, making use of large-scale synthetic datasets and achieving signiﬁcant improvements in terms of performance and speed (Wei et al. 2018; Guo et al. 2020; Lim and Kim 2020; Jiang et al. 2021; Li, Guo, and Loy 2021; Liu et al. 2021b).”
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation (2021, methodology): “LightenNet [39], MBLLEN [23]), and one GAN-based method (EnlightenGAN [9]).”
Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement (2020, methodology): “CNN-based models usually use self-captured paired data for network training [27, 5, 24] while GAN-based models elaborately select unpaired data [10].”
Low-Light Image Enhancement with Wavelet-Based Diffusion Models (2023, methodology): “In row 3, Zero-DCE, EnlightenGAN, and DDIM smooth the cloud texture, as Uformer and WeatherDiff produce gridding effects and messy content.”
SNR-Aware Low-light Image Enhancement (2022, methodology): “612 Methods LPNet [26] MIR-Net [59] RF [24] 3DLUT [60] A3DLUT [42] Band [52] EG [20] Retinex [29] Sparse [53] IPT [6] Uformer [44] Ours PSNR 19.”
R2RNet: Low-light Image Enhancement via Real-low to Real-normal Network (2021, background): “[23] proposed a network based on GAN for low-light image enhancement and used unpaired images for training for the first time.”
From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement (2020, methodology): “…Enhancement and Denoising Method (JED) [25], Reﬁned Retinex Model (RRM) [18], Simultaneous Reﬂectance and Illumination Estimation (SRIE) [8], Deep Retinex Decomposition (DRD) [30], Deep Underexposed Photo Enhancement (DeepUPE) [27], Single Image Contrast Enhancer (SICE) [2], EnlightenGAN [11].”