Deep learning enables cross-modality super-resolution in fluorescence microscopy

By Hongda Wang, Yair Rivenson, Yiyin Jin, Zhensong Wei, R. Gao, Harun Günaydın, L. Bentolila, C. Kural, Aydogan Ozcan. Nature Methods, 2018

713 citations recorded by the source corpus.

We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this framework, we improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching the resolution that is acquired using high-numerical-aperture objectives. We also demonstrate cross-modality super-resolution, transforming confocal microscopy images to match the resolution acquired with a stimulated emission depletion (STED) microscope. We further demonstrate that total internal reflection fluorescence (TIRF) microscopy images of subcellular structures within cells and tissues can be transformed to match the results obtained with a TIRF-based structured illumination microscope. The deep network rapidly outputs these super-resolved images, without any iterations or parameter search, and could serve to democratize super-resolution imaging. Deep learning enables cross-modality super-resolution imaging, including confocal-to-STED and TIRF-to-TIRF-SIM image transformation. Imaging of a larger FOV and greater depth of field is possible with higher resolution and SNR at lower light doses.

Subfields: Biomedical Engineering, Biophysics

View Paper: https://www.semanticscholar.org/paper/ce6454d4d63da26e2bab21044409b89d67914078

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Shanhui Fan
2. Demetri Psaltis
3. Cornelia Denz
4. David A. B. Miller
5. Sylvain Gigan
6. Aydogan Ozcan

In Biophysics

1. Gordon Wetzstein
2. Bernhard Kainz
3. Sylvain Gigan

Citations in the corpus

• A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis (2019, methodology): “Zheng et al. (2015) propose CRF-CNN, a recurrent neural network which has the desirable properties of both CNNs and CRFs. Wang et al. (2019) propose DeepIGeoS, an interactive geodesic framework for medical image segmentation.”
• Inference in artificial intelligence with deep optics and photonics (2020, background): “Data-driven approaches based on deep learning have been providing an alternative route for solving inverse problems in optical microscopy 72,79–86 .”