GANs for Biological Image Synthesis

By Anton Osokin, Anatole Chessel, Rafael Edgardo Carazo-Salas, Federico Vaggi. 2017

114 citations recorded by the source corpus.

In this paper, we propose a novel application of Generative Adversarial Networks (GAN) to the synthesis of cells imaged by fluorescence microscopy. Compared to natural images, cells tend to have a simpler and more geometric global structure that facilitates image generation. However, the correlation between the spatial pattern of different fluorescent proteins reflects important biological functions, and synthesized images have to capture these relationships to be relevant for biological applications. We adapt GANs to the task at hand and propose new models with casual dependencies between image channels that can generate multichannel images, which would be impossible to obtain experimentally. We evaluate our approach using two independent techniques and compare it against sensible baselines. Finally, we demonstrate that by interpolating across the latent space we can mimic the known changes in protein localization that occur through time during the cell cycle, allowing us to predict temporal evolution from static images.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, methodology): “These can also be used for biological image synthesis [207,208] and text-to-image synthesis [209–211].”
Feedback GAN (FBGAN) for DNA: a Novel Feedback-Loop Architecture for Optimizing Protein Functions (2018, methodology): “GANs have also been used to generate images of cells imaged by ﬂuorescent microscopy, uniquely using a separable generator where one channel of the image was used as input to generate another channel [17].”
Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network (2018, background): “To synthesize medical images, recent advances [24, 6] have used generative adversarial networks (GANs) [10] to formulate it as an image-to-image translation task.”
CytoGAN: Generative Modeling of Cell Images (2017, background): “GANs have been proven to synthesize realistic images both within and beyond the biological domain [7, 17, 14].”
GAN-based synthetic brain MR image generation (2018, background): “Differently from a very recent work of GANs for biological image synthesis (fluorescence microscopy) [19], to the best of our knowledge, this is the first GAN-based realistic brain tumor MR image generation approach aimed at data augmentation and physician training.”
Building a 3D Integrated Cell (2017, background): “These other GAN approaches (Osokin et al., 2017; Goldsborough et al., 2017), allow neither a probabilistic interpretation nor prediction of structure localization, in real images.”
Style Aggregated Network for Facial Landmark Detection (2018, background): “Following that, many researchers devoted great efforts to improve this research topic regarding theory [2, 8, 25, 35, 54] and applications [36, 41, 50, 71].”
Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction (2018, background): “Recently, much effort has been invested in building generative models of cells 90 using adversarial generative models 91–93 .”
Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training (2017, methodology): “[30] use GANs for synthesizing biological cells imaged by fluorescence microscopy.”