Learning to See in the Dark

By Cheng Chen, Qifeng Chen, Jia Xu, Vladlen Koltun. 2018

1,542 citations recorded by the source corpus.

Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To support the development of learning-based pipelines for low-light image processing, we introduce a dataset of raw short-exposure low-light images, with corresponding long-exposure reference images. Using the presented dataset, we develop a pipeline for processing low-light images, based on end-to-end training of a fully-convolutional network. The network operates directly on raw sensor data and replaces much of the traditional image processing pipeline, which tends to perform poorly on such data. We report promising results on the new dataset, analyze factors that affect performance, and highlight opportunities for future work.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CNN-Generated Images Are Surprisingly Easy to Spot… for Now (2019, methodology): “ProGAN [19] StyleGAN [20] BigGAN [7] CycleGAN [48] StarGAN [10] GauGAN [29] CRN [9] IMLE [23] SITD [8] Super-res.”
SNR-Aware Low-light Image Enhancement (2022, methodology): “…collection of SOTA methods for low-light image enhancement, including Dong [9], LIME [15], MF [11], SRIE [12], BIMEF [55], DRD [45], RRM [28], SID [5], DeepUPE [40], KIND [61], DeepLPF [33], FIDE [48], LPNet [26], MIR-Net [59], RF [24], 3DLUT [60], A3DLUT [42], Band [52], EG [20], Retinex [29],…”
Guided Collaborative Training for Pixel-wise Semi-Supervised Learning (2020, background): “Night Image Enhancement [8,13] is another common vision application.”
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, methodology): “Subsequent work on both single [5, 9] and multi-image [8, 19, 38, 53] denoising demonstrated the benefits of training networks to operate directly on noisy raw input data.”
Getting to Know Low-light Images with The Exclusively Dark Dataset (2018, background): “Recently, a very closed dataset to ours is the See-in-the-Dark (SID) dataset [19] that aim to produce perceptually good images in low-light conditions.”
EnlightenGAN: Deep Light Enhancement Without Paired Supervision (2019, background): “It also avoids overﬁtting any speciﬁc data generation protocol or imaging device that previous works [15, 5, 16] implicitly rely on, hence leading to notably improved real-world generalization.”
U-Shape Transformer for Underwater Image Enhancement (2021, methodology): “Therefore, we will introduce other general enhancement techniques such as low-light boosting [4] for further improvement.”
Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation (2019, background): “[3] and [7] provide only RAW images, while [25] uses simple processing which may not be realistic.”
From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement (2020, methodology): “There are also works on deep-learning based image enhancement from raw images [4], or the joint task of low-light image enhancement and high-level computer vision tasks, such as face detection [34], object detection [19], etc .”
Benchmarking Low-Light Image Enhancement and Beyond (2021, methodology): “2017)1 8000/2000 Synthetic UCID/BSD/Google SID (Chen et al. 2018) 5094 (RAW images) Real Camera SICE (Cai et al.”
Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset (2019, methodology): “As for extremely lowlight scenes, SID [10] develops a CNN-based pipeline to directly process raw sensor images.”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, methodology): “The encoder-decoder models [1], [2], [3], [4] first progressively map the input to a lowresolution representation, and then apply a gradual reverse mapping to the original resolution.”
GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing (2019, methodology): “Multi-scale estimation [31, 2] tackles this problem by i) approximating the high-dimensional statistical model by a low-dimensional one, ii) estimating the parameters of the low-dimensional model based on the training data, iii) parameterizing the neighborhood of the estimated low-dimensional model,”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, methodology): “The encoder-decoder models [84,59,17,124] first progressively map the input to a low-resolution representation, and then apply a gradual reverse mapping to the original resolution.”
Deep Learning Techniques for Inverse Problems in Imaging (2020, background): “For example, deep learning enables exceptional low-light imaging [30], as shown in Figure 3.”
Artificial intelligence in the creative industries: a review (2020, methodology): “[131] have developed specific low light denoising methods using CNN-based methods; Lempitsky et al.”
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation (2019, background): “which learns jointly from stylized images and unlabeled real images of increasing darkness and exploits the prior knowledge from a map. There is a rich literature on low-light image enhancement [27], [28], [29], which is also relevant to our work. However, its focus is more on the low-level goal of ”
URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement (2022, background): “Due to these limitations existing in model-based methods, researchers take advantage of deep networks [3,11,15, 25,34,42] to restore low-light images in a data-driven manner.”
Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study (2019, background): “Therefore, many newer and larger datasets are presented for image and video restoration, such as DIV2K [54] and MANGA109 [55] for image super-resolution, PolyU [56] and Darmstadt [57] for denoising, RawInDark [58] and LOL dataset [59] for low light enhancement, HazeRD [60], OHAZE [61] and IHAZE [62]”