HDR image reconstruction from a single exposure using deep CNNs

By Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafał K. Mantiuk, Jonas Unger. ACM Transactions on Graphics, 2017

626 citations recorded by the source corpus.

Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that have been lost in saturated image areas, in order to enable HDR reconstruction from a single exposure. We show that this problem is well-suited for deep learning algorithms, and propose a deep convolutional neural network (CNN) that is specifically designed taking into account the challenges in predicting HDR values. To train the CNN we gather a large dataset of HDR images, which we augment by simulating sensor saturation for a range of cameras. To further boost robustness, we pre-train the CNN on a simulated HDR dataset created from a subset of the MIT Places database. We demonstrate that our approach can reconstruct high-resolution visually convincing HDR results in a wide range of situations, and that it generalizes well to reconstruction of images captured with arbitrary and low-end cameras that use unknown camera response functions and post-processing. Furthermore, we compare to existing methods for HDR expansion, and show high quality results also for image based lighting. Finally, we evaluate the results in a subjective experiment performed on an HDR display. This shows that the reconstructed HDR images are visually convincing, with large improvements as compared to existing methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bdb142c8c89e12fcecd17d8824acef610f69febd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CycleISP: Real Image Restoration via Improved Data Synthesis (2020, background): “Similar to [21], the log loss term is added to enforce approximately equal treatment for all the image values; otherwise the network dedicates more attention to recovering the highlight regions.”
An Experiment-Based Review of Low-Light Image Enhancement Methods (2020, positive): “designed a deep convolutional neural network (DCNN) [264] based on a large dataset of HDR images, and Liu et al.”
Learning to Restore Low-Light Images via Decomposition-and-Enhancement (2020, methodology): “(G) HDRCNN [12] (H) DRHT [46] (I) DSLR [24] (J) SID [6] (K) JieP [5] (L) Ours”
ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content (2018, methodology): “Eilertsen et al. [EKD ∗ 17], use a U-Net like architecture to predict values for saturated areas of badly exposed content, whereas non-saturated areas are linearised by applying an inverse camera response curve.”
Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images (2018, background): “Recently, methods [28], [31] have been proposed to train a CNN network to map the low dynamic range (LDR) images to HDR images.”
End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging (2018, background): “For example, high dynamic range image estimation from a single low dynamic range photograph was recently demonstrated to achieve unprecedented image quality [Eilertsen et al. 2017; Kalantari and Ramamoorthi 2017; Zhang and Lalonde 2017].”
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, neutral): “Current approaches apply machine learning to produce HDR outputs from single [15] or multiple misaligned [24] LDR inputs, either recovering or hallucinating detail in clipped highlights.”
Attention-Guided Network for Ghost-Free High Dynamic Range Imaging (2019, methodology): “[3] proposed a deep autoencoder network to predict HDR values from one image.”
Deep reverse tone mapping (2017, methodology): “Considering the logarithm of pixel values might be helpful [Eilertsen et al. 2017] .”
Deep High Dynamic Range Imaging with Large Foreground Motions (2017, methodology): “rejection method [19], the flow-based method with CNN merger [14], and the single image HDR imaging [3].”