Latent Space Manipulation for High-Resolution Medical Image Synthesis via the StyleGAN.

By L. Fetty, M. Bylund, P. Kuess, G. Heilemann, T. Nyholm, D. Georg, Tommy Löfstedt. 2020

84 citations recorded by the source corpus.

INTRODUCTION This paper explores the potential of the StyleGAN model as an high-resolution image generator for synthetic medical images. The possibility to generate sample patient images of different modalities can be helpful for training deep learning algorithms as e.g. a data augmentation technique. METHODS The StyleGAN model was trained on Computed Tomography (CT) and T2- weighted Magnetic Resonance (MR) images from 100 patients with pelvic malignancies. The resulting model was investigated with regards to three features: Image Modality, Sex, and Longitudinal Slice Position. Further, the style transfer feature of the StyleGAN was used to move images between the modalities. The root-mean-squard error (RMSE) and the Mean Absolute Error (MAE) were used to quantify errors for MR and CT, respectively. RESULTS We demonstrate how these features can be transformed by manipulating the latent style vectors, and attempt to quantify how the errors change as we move through the latent style space. The best results were achieved by using the style transfer feature of the StyleGAN (58.7 HU MAE for MR to CT and 0.339 RMSE for CT to MR). Slices below and above an initial central slice can be predicted with an error below 75 HU MAE and 0.3 RMSE within 4cm for CT and MR, respectively. DISCUSSION The StyleGAN is a promising model to use for generating synthetic medical images for MR and CT modalities as well as for 3D volumes.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “In addition to common applications in image processing, in the last few months, GAN inversion techniques have been widely introduced to many other tasks, such as 3D reconstruction [139], [140], image understanding [141], [142], multimodal learning [133], [134], [135], [143], [144], and medical imagi”
Manipulating Medical Image Translation with Manifold Disentanglement (2020, methodology): “The closest use of a style-based GAN in medical imaging analysis is [36], which uses the original StyleGAN to explore the latent space of medical images.”
A review of medical image data augmentation techniques for deep learning applications (2021, methodology): “To overcome these challenges, many previously proposed GAN variants, with improved network architecture and mathematical optimisation, have been adopted in Ref.53,87,91,131 – 140 as listed in Table 1.”