Noise2Noise: Learning Image Restoration without Clean Data
By Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, Timo Aila. 2018
2,082 citations recorded by the source corpus.
We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: under certain common circumstances, it is possible to learn to restore signals without ever observing clean ones, at performance close or equal to training using clean exemplars. We show applications in photographic noise removal, denoising of synthetic Monte Carlo images, and reconstruction of MRI scans from undersampled inputs, all based on only observing corrupted data.
Subfields: Computer Vision and Pattern Recognition, Geophysics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ming-Hsuan Yang
- Dacheng Tao
- Chen Change Loy
- Jitendra Malik
- Xiaogang Wang
- Trevor Darrell
- Andrea Vedaldi
In Geophysics
Citations in the corpus
- Elucidating the Design Space of Diffusion-Based Generative Models (2022, background): “This suggests that a major component of the hypothesized non-conservativity of D θ ( x ; σ ) is a tendency to remove slightly too much noise — most likely due to regression toward the mean that can be expected to happen with any L 2 -trained denoiser [29].”
- Seeing Motion in the Dark (2019, background): “[26] showed that a denoising network can be trained without clean ground-truth if the noise is unbiased.”
- Adding Conditional Control to Text-to-Image Diffusion Models (2023, background): “Manipulating the initial convolution weights is also discussed in ProGAN [21] and StyleGAN [22], as well as Noise2Noise [33] and [65].”
- Learning a Simple Low-Light Image Enhancer from Paired Low-Light Instances (2023, background): “As discussed in [16], paired noisy images can be leveraged to train denoising models.”
- NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, methodology): “Noise2Noise [32] demonstrated this was possible given a dataset of pairs of independent noisy observations of the same image, an insight Ehret et al.”
- Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images (2021, methodology): “The summary of Section 3 is as follows: In Section 3.1, we revisit the re-lated theory proposed in Noise2Noise, which proves that paired noisy images taken from the same scene can also be used to train denoising models.”
- High-Quality Self-Supervised Deep Image Denoising (2019, methodology): “[17] introduced NOISE2NOISE training, where pairs of corrupted images are used as training data.”
- You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network (2020, methodology): “example, Noise2Noise (N2N) (Lehtinen et al. 2018) shows that simple statistical arguments lead to new capabilities in data denoising using neural networks.”
- Deep Surface Normal Estimation With Hierarchical RGB-D Fusion (2019, background): “will learn an expectation of all pairs of input and groundtruth [13]:”
- Noise2Void - Learning Denoising From Single Noisy Images (2018, background): “With N2V we have to stick to the more narrow task of denoising, as we rely on the fact that multiple noisy observations can help us to retrieve the true signal [12].”
- A Survey of Deep Learning for Scientific Discovery (2020, methodology): “For instance, Noise2Noise [122] uses a U-net neural network architecture to denoise images given multiple noisy copies.”
- AdaBins: Depth Estimation Using Adaptive Bins (2020, background): “Encoder-decoder networks have made significant contributions in many vision related problems such as image segmentation [35], optical flow estimation [10], and image restoration [28].”
- Unprocessing Images for Learned Raw Denoising (2018, background): “Recent work suggests that multiple noisy images of the same scene can be used as training data instead of paired noisy and noisefree images [25], but this does not substantially mitigate the”
- Isotropic reconstruction for electron tomography with deep learning (2022, background): “The Noise2Noise strategy 36 (i.e., learning from one noisy instance to another noisy instance of an image) in these methods requires no noise-free ground truth and only makes assumptions on the independency and zero-mean statistics of the noise.”
- Non-uniform refinement: adaptive regularization improves single-particle cryo-EM reconstruction (2019, background): “Similar objectives have been used for image de-noising [18].”
- Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives (2020, background): “Noise2Noise [204] is an elegant example of unsupervised denoising where the authors learn denoised data without clean data.”
- Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, methodology): “It has been shown, both theoretically and empirically, that in such applications one can train a deep learning model using the noisy signals as target, meaning that one only needs noisy labels [144].”