GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification

By Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Marianne Amitai, Jacob Goldberger, Hayit Greenspan. 2018

1,883 citations recorded by the source corpus.

Abstract Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classification. Our novel method is demonstrated on a limited dataset of computed tomography (CT) images of 182 liver lesions (53 cysts, 64 metastases and 65 hemangiomas). We first exploit GAN architectures for synthesizing high quality liver lesion ROIs. Then we present a novel scheme for liver lesion classification using CNN. Finally, we train the CNN using classic data augmentation and our synthetic data augmentation and compare performance. In addition, we explore the quality of our synthesized examples using visualization and expert assessment. The classification performance using only classic data augmentation yielded 78.6% sensitivity and 88.4% specificity. By adding the synthetic data augmentation the results increased to 85.7% sensitivity and 92.4% specificity. We believe that this approach to synthetic data augmentation can generalize to other medical classification applications and thus support radiologists’ efforts to improve diagnosis.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Generative Models

Citations in the corpus

ScaleNet: An Unsupervised Representation Learning Method for Limited Information (2023, methodology): “This experiment clearly shows the adjustment of the ScaleNet to understand and detect influential features with limited information, that will be crucial for many fields with a lack of data [13,33,42].”
GAN Memory with No Forgetting (2020, background): “…consequently also protecting privacy (the original data need not be saved); and ( iii ) a GAN often generates realistic samples not observed in training data, delivering a synthetic data augmentation that potentially beneﬁts better performance of downstream tasks [78, 8, 9, 21, 33, 26, 27].”
Generative Adversarial Network in Medical Imaging: A Review (2018, methodology): “Frid-Adar et al. (2018) DCGAN /ACGAN – M30 [Liver lesion] Generating each lesion class separately (with DCGAN) is than generating all classes at once (using ACGAN)”
On Leveraging Pretrained GANs for Limited-Data Generation (2020, methodology): “…data augmentation like shift, zooming, rotation, or flipping, GANs trained on limited data can be used to do synthetic augmentation like transformed-style or fake observation-label/segmentation pair (Wang & Perez, 2017; Bowles et al., 2018; Frid-Adar et al., 2018; Han et al., 2019; 2020).”
A Gentle Introduction to Deep Learning in Medical Image Processing (2018, background): “In most recent literature, reports are found that also GANs are useful for data augmentation [59].”
Large-Scale Generative Data-Free Distillation (2020, methodology): “A typical example is medical imaging records [12, 20], whose availability is time-limited due to security and confidentiality.”
A Comprehensive Survey of Image Augmentation Techniques for Deep Learning (2022, background): “Model-based Unconditional DCGAN [41], [42, 43, 44] Label-conditional BDA [45], ImbCGAN [46], BAGAN [47], DAGAN [48], MFC-GAN [49], IDA-GAN [50].”
Synthetic Data for Deep Learning (2019, methodology): “[180, 181] used two standard GAN architectures, Deep Convolutional GAN (DCGAN) [472] and Auxiliary Classifier GAN (ACGAN) [428] with class label auxiliary information, to generate synthetic computed tomography (CT) images of liver lesions.”
DARE: Data Augmented Relation Extraction with GPT-2 (2020, background): “Oversampling approaches for textual data have been somehow limited as opposed to those for image data (Wong et al., 2016; Fawzi et al., 2016; Wang and Perez, 2017; Frid-Adar et al., 2018), since text semantics depend inherently on the exact order or structure of word tokens.”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, background): “In [37], for example, synthetic image augmentation based on the generative adversarial network (GAN) was shown to generate lesion image samples that were not recognized as synthetic by the expert radiologists and also increased CNN performance in classifying liver lesions.”
Data Augmentation in Classification and Segmentation: A Survey and New Strategies (2023, methodology): “, 2018 [72] Liver lesions dataset Translation, rotation, scaling, flipping and shearing, and GAN-based synthetic images.”
Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks (2018, methodology): “liver lesion classification in synthetic CT [79], and chest pathology classification in synthetic X-ray images [80].”