DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks

By Andrey D. Ignatov, Nikolay Kobyshev, Kenneth Vanhoey, Radu Timofte, Luc Van Gool. 2017

585 citations recorded by the source corpus.

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations – small sensor size, compact lenses and the lack of specific hardware, – impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images. We propose learning the translation function using a residual convolutional neural network that improves both color rendition and image sharpness. Since the standard mean squared loss is not well suited for measuring perceptual image quality, we introduce a composite perceptual error function that combines content, color and texture losses. The first two losses are defined analytically, while the texture loss is learned in an adversarial fashion. We also present DPED, a large-scale dataset that consists of real photos captured from three different phones and one high-end reflex camera. Our quantitative and qualitative assessments reveal that the enhanced image quality is comparable to that of DSLR-taken photos, while the methodology is generalized to any type of digital camera.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (2021, background): “types and the real RealSRSet which consists of 20 real images either downloaded from the internet or directly chosen from existing testing datasets [17, 32, 33, 52].”
Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception (2020, methodology): “Such feature preservation has been found to be very effective for image enhancement, style transfer, and SISR problems [31, 34]; as suggested in [38], we define the image content function ΦV GG(·) as high-level features extracted by the last conv layer of a pretrained VGG-19 network.”
Rethinking Coarse-to-Fine Approach in Single Image Deblurring (2021, background): “Recent studies also suggest the auxiliary loss terms in addition to the content loss for the performance improvement [11, 9].”
ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting (2023, methodology): “It comprises 35 LR images widely used in recent literature [19, 58, 59, 60, 61].”
FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing (2020, background): “With the high-frequency details removed, LF stresses the brightness, color and contrast information of images, and it can make color comparison easier (Ignatov et al. 2017).”
Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset (2019, methodology): “Besides, DPED [36,70] proposes an end-to-end approach using a composite perceptual error function for translating low-quality mobile phone photos into DSLRquality photos.”
Frequency Separation for Real-World Super-Resolution (2019, methodology): “Experiments on Real-World Images We also evaluate our method on real-world images from the DPED dataset [12], where we use the train images captured by an iPhone 3 camera for ﬁne-tuning.”
Exploiting Diffusion Prior for Real-World Image Super-Resolution (2023, unknown): “More results are provided in the supplementary material.”
Unsupervised Learning for Real-World Super-Resolution (2019, methodology): “We also perform experiments on the DPED [17] dataset, acquired by a smartphone camera.”
Fast Underwater Image Enhancement for Improved Visual Perception (2019, methodology): “Energy-based GANs [27] also improve training stability by modeling the discriminator as an energy function, whereas the Least-Squared GAN [28] addresses the vanishing gradients problem by adopting a least-square loss function for the discriminator.”
An Experiment-Based Review of Low-Light Image Enhancement Methods (2020, methodology): “They built a dataset consisting of images of the same scene taken by the different cameras [262] and presented an end-to-end deep learning approach for translating ordinary photos into DSLR-quality images.”
AI Benchmark: Running Deep Neural Networks on Android Smartphones (2018, methodology): “problem that encompasses various kinds of improvements including color enhancement, denoising, sharpening, texture synthesis, etc. In this formulation the problem was ﬁrst addressed in the DPED paper [4], where 8 the authors were trying to turn low-quality smartphone photos into photos as they would”
Image-to-Image Translation: Methods and Applications (2021, background): “Image quality improvement [2], [207], [208], [210], [209] focuses on producing noticeably fewer colored artifacts around hard edges and more accurate colors, as well as reduced noise in smooth shadows.”
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs (2018, methodology): “(1) DPED [10]: since we only evaluate on generators, we took only the generator of their GAN architecture.”
Underexposed Photo Enhancement Using Deep Illumination Estimation (2019, background): “More recent work [16, 17, 13, 9, 15, 22] takes data-driven approaches to simultaneously learn adjustment in terms of color, contrast, brightness, and saturation for producing more expressive results.”
NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results (2020, methodology): “As a ground truth to super-resolved images above sensor size does not exist, we use crops of the validation set of DPED for a human perception study.”
Low-Light Image Enhancement via a Deep Hybrid Network (2019, methodology): “[4] learn a mapping between photos from mobile devices and a DSLR camera based on an end-to-end residual”
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data (2021, methodology): “We test on several testing datasets with real-world images, including RealSR [5], DRealSR [49], OST300 [47], DPED [17], ADE20K validation [57] and Internet images.”
PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN With Dual-Discriminators (2023, methodology): “[28], [29] have achieved surprising performance in improving perceptual image quality from an extensive collection of paired or unpaired data.”
Real-World Super-Resolution via Kernel Estimation and Noise Injection (2020, methodology): “DPED The DPED [15] dataset contains 5, 614 images taken by the iPhone3 camera.”