Deep learning ensembles for melanoma recognition in dermoscopy images

By Noel C. F. Codella, Q. Nguyen, Sharath Pankanti, David Gutman, Brian Helba, Allan Halpern, John R. Smith. 2016

600 citations recorded by the source corpus.

Melanoma is the deadliest form of skin cancer. While curable with early detection, only highly trained specialists are capable of accurately recognizing the disease. As expertise is in limited supply, automated systems capable of identifying disease could save lives, reduce unnecessary biopsies, and reduce costs. Toward this goal, we propose a system that combines recent developments in deep learning with established machine learning approaches, creating ensembles of methods that are capable of segmenting skin lesions, as well as analyzing the detected area and surrounding tissue for melanoma detection. The system is evaluated using the largest publicly available benchmark dataset of dermoscopic images, containing 900 training and 379 testing images. New state-of-the-art performance levels are demonstrated, leading to an improvement in the area under receiver operating characteristic curve of 7.5% (0.843 vs. 0.783), in average precision of 4% (0.649 vs. 0.624), and in specificity measured at the clinically relevant 95% sensitivity operating point 2.9 times higher than the previous state-of-the-art (36.8% specificity compared to 12.5%). Compared to the average of 8 expert dermatologists on a subset of 100 test images, the proposed system produces a higher accuracy (76% vs. 70.5%), and specificity (62% vs. 59%) evaluated at an equivalent sensitivity (82%).

Subfields: Artificial Intelligence, Biomedical Engineering, Oncology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Oncology

Citations in the corpus

Opportunities and obstacles for deep learning in biology and medicine (2017, background): “melanoma task, reported performance was competitive with or better than a board of certified dermatologists [4,47].”
Automated detection of COVID-19 cases using deep neural networks with X-ray images (2020, methodology): “Deep learning techniques have been successfully applied in many problems such as arrhythmia detection [28 – 30], skin cancer classification [31,32], breast cancer detection [33,34], brain disease classification [35], pneumonia detection from chest X-ray images [36], fundus image segmentation [37],…”
Skin lesion classification from dermoscopic images using deep learning techniques (2017, methodology): “[10] utilize hand-coded feature extraction techniques including color histogram, edge histogram, and a multi-scale variant of color local binary patterns (LBP).”
Dermatologist-Level Classification of Skin Cancer Using Cascaded Ensembling of Convolutional Neural Network and Handcrafted Features Based Deep Neural Network (2022, methodology): “Codella et al. [22] have proposed a high accuracy using ConvNet to extract image descriptors by using a pre-trained model from skin lesion image.”
Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC) (2019, methodology): “Prior work measured the average Jaccard between 3 expert annotators on 100 images from the 2016 challenge [6].”
Skin lesion analysis toward melanoma detection: A challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC) (2016, methodology): “This ﬁnding has spawned an interest in building techniques for automated assessment of dermoscopic images [11, 12, 13, 14].”
Classification of the Clinical Images for Benign and Malignant Cutaneous Tumors Using a Deep Learning Algorithm. (2018, background): “Several dermatologic studies have reported on the use of deep learning or machine learning (Binder et al., 1994; Codella et al., 2017; Esteva et al., 2017; Liao, 2016).”
Results of the 2016 International Skin Imaging Collaboration ISBI Challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images (2017, methodology): “15,17,18 In December 2015, 1,552 lesions were chosen from ~12,000 dermoscopic images in the ISIC Archive; after excluding 273 for inadequate image quality, 1,279 lesions (248 (19.3”
Skin Lesion Analysis towards Melanoma Detection Using Deep Learning Network (2017, background): “In recent research, Codella and his colleagues established a system combining recent developments in deep learning and machine learning approaches for skin lesion segmentation and classification [30].”
Collaborative Human-AI (CHAI): Evidence-Based Interpretable Melanoma Classification in Dermoscopic Images (2018, neutral): “…snapshots of the dataset have been used to host two consecutive years of benchmark challenges [1,2], which have increased interest in the computer vision community [2,3,4,5,6], and supported the development of methods that surpassed the diagnostic performance of expert clinicians [2,3,4].”
Progressive Transfer Learning and Adversarial Domain Adaptation for Cross-Domain Skin Disease Classification (2020, methodology): “Recently, deep neural networks were used to analyze pathology and clinical images and showed greatsuccess[9]–[11],includingforskindiseaserecognition[7], [12]–[17],whichalleviatesthedrawbacksofclassicalmachine learningmethods.”
Visualizing Convolutional Neural Networks to Improve Decision Support for Skin Lesion Classification (2018, background): “In the subfield of dermatology, CNNs have been applied to the problem of skin lesion classification, based on dermoscopy images, where they set a new state-of-the-art benchmark, matching—or even surpassing—medical expert performance [2,3,5].”
A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. (2019, neutral): “…(2018)38 Liver fibrosis Histology Yes Resampling method Yes Ciompi et al (2017)39 Lung cancer Expert consensus Yes Random split sample validation Yes Codella et al (2017)40 Melanoma Histology No Random split sample validation No Coudray et al (2018)41 Lung cancer Histology Yes NR Yes De Fauw et…”
State-of-the-art review on deep learning in medical imaging. (2019, background): “[56] Dermatology (1279) Melanoma Segmentation FCN Acc: 76%”
A comprehensive review on ensemble deep learning: Opportunities and challenges (2023, background): “The authors of Codella et al. (2017) proposed voting based on residual networks (DRN) and CNNs for melanoma recognition in dermoscopy images.”