Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks
By V. Sandfort, Ke Yan, P. Pickhardt, Ronald M. Summers. 2019
650 citations recorded by the source corpus.
Labeled medical imaging data is scarce and expensive to generate. To achieve generalizable deep learning models large amounts of data are needed. Standard data augmentation is a method to increase generalizability and is routinely performed. Generative adversarial networks offer a novel method for data augmentation. We evaluate the use of CycleGAN for data augmentation in CT segmentation tasks. Using a large image database we trained a CycleGAN to transform contrast CT images into non-contrast images. We then used the trained CycleGAN to augment our training using these synthetic non-contrast images. We compared the segmentation performance of a U-Net trained on the original dataset compared to a U-Net trained on the combined dataset of original data and synthetic non-contrast images. We further evaluated the U-Net segmentation performance on two separate datasets: The original contrast CT dataset on which segmentations were created and a second dataset from a different hospital containing only non-contrast CTs. We refer to these 2 separate datasets as the in-distribution and out-of-distribution datasets, respectively. We show that in several CT segmentation tasks performance is improved significantly, especially in out-of-distribution (noncontrast CT) data. For example, when training the model with standard augmentation techniques, performance of segmentation of the kidneys on out-of-distribution non-contrast images was dramatically lower than for in-distribution data (Dice score of 0.09 vs. 0.94 for out-of-distribution vs. in-distribution data, respectively, p < 0.001). When the kidney model was trained with CycleGAN augmentation techniques, the out-of-distribution (non-contrast) performance increased dramatically (from a Dice score of 0.09 to 0.66, p < 0.001). Improvements for the liver and spleen were smaller, from 0.86 to 0.89 and 0.65 to 0.69, respectively. We believe this method will be valuable to medical imaging researchers to reduce manual segmentation effort and cost in CT imaging.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- D. Bull
- Nantheera Anantrasirichai
- P. Chlap
- L. Holloway
- A. Haworth
- Nym Vandenberg
- Onat Dalmaz
- Mahmut Yurt
In Computer Vision and Pattern Recognition
- Shekhar S. Chandra
- Stuart Crozier
- D. Bull
- P. Greer
- Nantheera Anantrasirichai
- C. Engstrom
- P. Chlap
- L. Holloway
In Generative Models
In Medical Imaging
Citations in the corpus
- Artificial intelligence in the creative industries: a review (2020, methodology): “This slightly modified architecture is trained to transform contrast CT images (computed tomography scans) into non-contrast images [308].”
- A review of medical image data augmentation techniques for deep learning applications (2021, methodology): “53,83,86,87,100,119 For future work, we believe basic and deformable augmentation will continue to be regularly used for training DL networks.”
- ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis (2021, methodology): “In recent years, GAN-based methods were demonstrated to offer state-of-the-art performance in numerous synthesis tasks, including data augmentation as well as multi-modal synthesis [24], [34], [71], [72].”
- Manipulating Medical Image Translation with Manifold Disentanglement (2020, methodology): “For example, [4] uses GAN-generated CT scans to improve segmentation accuracy, and [5] uses three generative networks to synthesise three types of lesions to improve classification accuracy.”
- SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining (2021, neutral): “In comparison, generative adaptation methods seek to match the source images to the target domain with image-to-image translation meth-ods [59, 34, 74].”