CycleISP: Real Image Restoration via Improved Data Synthesis

By Syed Waqas Zamir, Aditya Arora, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao. Computer Vision and Pattern Recognition, 2020

446 citations recorded by the source corpus.

The availability of large-scale datasets has helped unleash the true potential of deep convolutional neural networks (CNNs). However, for the single-image denoising problem, capturing a real dataset is an unacceptably expensive and cumbersome procedure. Consequently, image denoising algorithms are mostly developed and evaluated on synthetic data that is usually generated with a widespread assumption of additive white Gaussian noise (AWGN). While the CNNs achieve impressive results on these synthetic datasets, they do not perform well when applied on real camera images, as reported in recent benchmark datasets. This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline. In this paper, we present a framework that models camera imaging pipeline in forward and reverse directions. It allows us to produce any number of realistic image pairs for denoising both in RAW and sRGB spaces. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets. The parameters in our models are ~5 times lesser than the previous best method for RAW denoising. Furthermore, we demonstrate that the proposed framework generalizes beyond image denoising problem e.g., for color matching in stereoscopic cinema. The source code and pre-trained models are available at https://github.com/swz30/CycleISP.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/51352023ed22068fbfaa7693fc37123c0a5fce9f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

HINet: Half Instance Normalization Network for Image Restoration (2021, methodology): “For CycleISP, we set the initial learning rate to 2 × 10 − 4 with a batch size of 16 for 3 × 10 5 iterations.”
Uformer: A General U-Shaped Transformer for Image Restoration (2021, methodology): “We compare Uformer with 8 state-of-the-art denoising methods, including the featurebased BM3D [13] and seven learning-based methods: RIDNet [4], VDN [73], CycleISP [75], NBNet [9], DANet [74], MIRNet [76], and MPRNet [77].”
Restormer: Efficient Transformer for High-Resolution Image Restoration (2021, unknown): “Method DnCNN BM3D CBDNet* RIDNet* AINDNet* VDN SADNet* DANet+* CycleISP* MIRNet DeamNet* MPRNet DAGL Uformer Restormer Dataset [101] [15] [25] [6] [35] [89] [12] [90] [91] [92] [63] [93] [55] [80] (Ours)”
Learning Enriched Features for Real Image Restoration and Enhancement (2020, result): “Recently, deep learningbased approaches [11,5,9,35,39,80,120,121,119] make significant advances in image denoising, yielding favorable results than those of the hand-crafted methods.”
Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images (2021, methodology): “Furthermore, the performance of our approach can be further improved if we replace UNet [26] with a more advanced denoising architecture composed of multiple RRGs in [32].”
Deep Generalized Unfolding Networks for Image Restoration (2022, methodology): “Noisy RIDNet [3] DANet [76] CycleISP [77] DeamNet [53] MPRNet [79] DGUNet DGUNet”
MAXIM: Multi-Axis MLP for Image Processing (2022, methodology): “Visual examples for image denoising on SIDD [2] among VDN [104], DANet [105], MIRNet [107], CycleISP [106], MPRNet [108], and the proposed MAXIM-3S.”
Multi-Stage Progressive Image Restoration (2021, methodology): “Numerous network modules and functional units for image restoration have been developed including recursive residual learning [4, 95], dilated convolutions [4, 81], attention mechanisms [17, 86, 96], dense connections [73, 75, 97], encoder-decoders [7, 13, 43, 65], and generative models [44, 62, 90,”
Image De-Raining Transformer (2022, methodology): “On the other hand, our work primarily focuses on establishing a transformer-based method for single image de-TABLE [114] Model BM3D[115] DnCNN[38] VDN[116] DANet[117] CycleISP[118] MIRNet[119] DeamNet[120] MPRNet[79] DAGL[121] Uformer-S raining.”
Learning Enriched Features for Fast Image Restoration and Enhancement (2022, methodology): “32 dB over CycleISP [38] on SIDD and 0 .”