Generating Training Data for Denoising Real RGB Images via Camera Pipeline Simulation

By Ronnachai Jaroensri, Camille Biscarrat, Miika Aittala, Frédo Durand. 2019

29 citations recorded by the source corpus.

Image reconstruction techniques such as denoising often need to be applied to the RGB output of cameras and cellphones. Unfortunately, the commonly used additive white noise (AWGN) models do not accurately reproduce the noise and the degradation encountered on these inputs. This is particularly important for learning-based techniques, because the mismatch between training and real world data will hurt their generalization. This paper aims to accurately simulate the degradation and noise transformation performed by camera pipelines. This allows us to generate realistic degradation in RGB images that can be used to train machine learning models. We use our simulation to study the importance of noise modeling for learning-based denoising. Our study shows that a realistic noise model is required for learning to denoise real JPEG images. A neural network trained on realistic noise outperforms the one trained with AWGN by 3 dB. An ablation study of our pipeline shows that simulating denoising and demosaicking is important to this improvement and that realistic demosaicking algorithms, which have been rarely considered, is needed. We believe this simulation will also be useful for other image reconstruction tasks, and we will distribute our code publicly.

Subfields: Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Graphics and Computer-Aided Design

1. Chunhua Shen
2. Hao Chen
3. Ying Li
4. Haokui Zhang

In Computer Vision and Pattern Recognition

1. Hao Chen
2. Chunhua Shen
3. Ying Li
4. Haokui Zhang
5. Luc Van Gool
6. Alan L. Yuille
7. Xiaogang Wang
8. Dacheng Tao

Citations in the corpus

• Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising (2019, methodology): “Similar work in [12] proposes a camera simulator that aims to accurately simulate the degradation and noise transformation performed by camera pipelines.”
• Deep Learning on Image Denoising: An overview (2019, unknown): “From the point of view of producing the noisy image, imitating cameral pipelines to construct the degradation model was very effective to filter the real noisy [80].”
• IR-NAS: Neural Architecture Search for Image Restoration (2019, methodology): “Similar work in [10] proposes a camera simulator that aims to accurately simulate the degradation and noise transformation performed by camera pipelines.”