Content-aware image restoration: pushing the limits of fluorescence microscopy

By Martin Weigert, Uwe Schmidt, Tobias Boothe, A. Müller, Alexandr Dibrov, Akanksha Jain, Benjamin Wilhelm, Deborah Schmidt, Coleman Broaddus, Siân Culley and 11 more. Nature Methods, 2018

1,132 citations recorded by the source corpus.

Fluorescence microscopy is a key driver of discoveries in the life sciences, with observable phenomena being limited by the optics of the microscope, the chemistry of the fluorophores, and the maximum photon exposure tolerated by the sample. These limits necessitate trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. In this work we show how content-aware image restoration based on deep learning extends the range of biological phenomena observable by microscopy. We demonstrate on eight concrete examples how microscopy images can be restored even if 60-fold fewer photons are used during acquisition, how near isotropic resolution can be achieved with up to tenfold under-sampling along the axial direction, and how tubular and granular structures smaller than the diffraction limit can be resolved at 20-times-higher frame rates compared to state-of-the-art methods. All developed image restoration methods are freely available as open source software in Python, FIJI, and KNIME. Content-aware image restoration (CARE) uses deep learning to improve microscopy images. CARE bypasses the trade-offs between imaging speed, resolution, and maximal light exposure that limit fluorescence imaging to enable discovery.

Subfields: Biophysics, Media Technology

View Paper: https://www.semanticscholar.org/paper/6ff93aa6ca902002d16fb0c2d3fb48aead92c61e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Media Technology

Citations in the corpus

Label-free prediction of three-dimensional fluorescence images from transmitted light microscopy (2018, background): “While many strategies exist to minimize this photodamage (i.e. oxygen scavenging, reduced laser power and exposure, advanced microscopy techniques 2 , machine learning driven denoising 8 ), all present compromises with respect to ease, image quality, information content, and fidelity.”
A Survey of Deep Learning for Scientific Discovery (2020, background): “Two recent papers look at U-nets for super-resolution of ﬂuorescence microscopy [245] (code: https://csbdeep . bioimagecomputing . com/ ) and electron microscopy [56].”
High-performance medicine: the convergence of human and artificial intelligence (2019, background): “This is all the more reason that systematic debugging, audit, extensive simulation, and validation, along with prospective scrutiny, are required when an AI algorithm is unleashed in clinical practice.”
The ImageJ ecosystem: Open‐source software for image visualization, processing, and analysis (2020, background): “61 Initially, Weigert 35 Their work also demonstrated how to resolve granular structures smaller than the diffraction limit at 20-times-higher frame rates.”
Deep-STORM: super-resolution single-molecule microscopy by deep learning (2018, background): “Here we demonstrate precise, fast, parameter-free, super-resolution image reconstruction by harnessing Deep-Learning.”
Noise2Void - Learning Denoising From Single Noisy Images (2018, methodology): “We then compare our results to the ones obtained by a traditionally trained network [24], a N2N trained network and several self-supervised methods like BM3D [5], non-local means [2], and to mean-and median-ﬁlters.”