Noise2Self: Blind Denoising by Self-Supervision

By Joshua D. Batson, Loic A. Royer. International Conference on Machine Learning, 2019

855 citations recorded by the source corpus.

We propose a general framework for denoising high-dimensional measurements which requires no prior on the signal, no estimate of the noise, and no clean training data. The only assumption is that the noise exhibits statistical independence across different dimensions of the measurement, while the true signal exhibits some correlation. For a broad class of functions ("$\mathcal{J}$-invariant"), it is then possible to estimate the performance of a denoiser from noisy data alone. This allows us to calibrate $\mathcal{J}$-invariant versions of any parameterised denoising algorithm, from the single hyperparameter of a median filter to the millions of weights of a deep neural network. We demonstrate this on natural image and microscopy data, where we exploit noise independence between pixels, and on single-cell gene expression data, where we exploit independence between detections of individual molecules. This framework generalizes recent work on training neural nets from noisy images and on cross-validation for matrix factorization.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/ea9cf47573638745c9992cf9c5ebdabadd3c6849

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Cellpose3: one-click image restoration for improved cellular segmentation (2024, positive): “We compared the performance of the Cellpose3 models to Noise2Void, Noise2Self, and CARE [7, 14, 15].”
A Survey of Deep Learning for Scientific Discovery (2020, methodology): “The recent Noise2Self [14] (with code: https://github . com/czbiohub/noise2self ) frames denoising as a self-supervision problem, using diﬀerent subsets of features (with assumed independent noise properties) to perform denoising, applying it to both images as well as other high dimensional data.”
Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images (2021, methodology): “Subsequently, self-supervised denoising models, including Noise2Void [13] and Noise2Self [3], were proposed to train the networks only with one noisy observation per scene.”
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images (2021, background): “Various followups to Noise2Noise proposed modified network architectures allowing supervision with a dataset of single noisy images [4, 30, 31].”
Self2Self With Dropout: Learning Self-Supervised Denoising From Single Image (2020, methodology): “Our method is also compared to several recent dataset-based deep learning methods with published training codes, including N2V [15], N2S [3], N2N [19] and (C)DnCNN [31].”
Plug-and-Play Image Restoration With Deep Denoiser Prior (2020, background): “In [50], [51], [52], a novel training strategy without ground-truth is proposed.”
Deep Learning in Medical Imaging (2019, methodology): “83 Batson and Royer 84 used the concept of J-invariant and designed the Noise-2Self concept.”
Noisier2Noise: Learning to Denoise From Unpaired Noisy Data (2019, background): “Both [8] and [1] propose to learn to perform denoising using only a single noisy realization of each training sample.”
Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction (2018, background): “Self-supervision approaches such as noise2noise 34,35 , noise2self 36 , and noise2void 37,38 leverage statistical independence of noise across pixels.”
Self-Supervised Learning of Graph Neural Networks: A Unified Review (2021, background): “In addition to contrastive models such as [49], attributes masking is also commonly applied in predictive models [3], [6] for regularized reconstruction.”
Noise2Void - Learning Denoising From Single Noisy Images (2018, methodology): “In concurrent work [1], Batson et al . also introduce a method for self-supervised training of neural networks and other systems that is based on the idea of removing parts of the input.”
High-Quality Self-Supervised Deep Image Denoising (2019, methodology): “Two masking strategies were evaluated: copying from another pixel in a 5×5 neighborhood (denoted COPY) as advocated in [14], and overwriting the pixel with a random color in [0, 1]3 (denoted RANDOM), as done by Batson and Royer [1].”