Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
By Noel C. F. Codella, V. Rotemberg, P. Tschandl, M. Emre Celebi, S. Dusza, David Gutman, Brian Helba, Aadi Kalloo, K. Liopyris, M. Marchetti and 2 more. 2019
1,802 citations recorded by the source corpus.
This work summarizes the results of the largest skin image analysis challenge in the world, hosted by the International Skin Imaging Collaboration (ISIC), a global partnership that has organized the world's largest public repository of dermoscopic images of skin. The challenge was hosted in 2018 at the Medical Image Computing and Computer Assisted Intervention (MICCAI) conference in Granada, Spain. The dataset included over 12,500 images across 3 tasks. 900 users registered for data download, 115 submitted to the lesion segmentation task, 25 submitted to the lesion attribute detection task, and 159 submitted to the disease classification task. Novel evaluation protocols were established, including a new test for segmentation algorithm performance, and a test for algorithm ability to generalize. Results show that top segmentation algorithms still fail on over 10% of images on average, and algorithms with equal performance on test data can have different abilities to generalize. This is an important consideration for agencies regulating the growing set of machine learning tools in the healthcare domain, and sets a new standard for future public challenges in healthcare.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Oncology, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Jianfeng Gao
- Andrew Zisserman
- Jitendra Malik
In Oncology
- Noel C. F. Codella
- Allan Halpern
- Brian Helba
- John R. Smith
- Huan Fu
- Michael Alexander Riegler
- Gary S. Collins
- R. Shi
In Medical Imaging
- Geert J. S. Litjens
- Ben Glocker
- Huan Fu
- R. Shi
- Anne L. Martel
- Noel C. F. Codella
- Jiancheng Yang
- Bingbing Ni
Citations in the corpus
- Florence: A New Foundation Model for Computer Vision (2021, background): “…2016) (plant leaf images, 38 disease states over 14 plant species), EuroSAT (Helber et al., 2019) (RGB satellite images, 10 categories), ISIC 2018 (Codella et al., 2019; Tschandl et al., 2018) (dermoscopic images of skin lesions, 7 disease states), and ChestX (Wang et al., 2017) (Chest X-rays, 16…”
- Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “In particular, a popular dataset from ISIC (International Skin Imaging Collaboration) has colorful patches present in approximately 50% of the non-cancerous images but not in the cancerous images (Codella et al., 2019).”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “However, this practice has not been ubiquitous due to the difficulty of obtaining data spanning multiple hospitals (Esteva et al., 2017; Bejnordi et al., 2017; Codella et al., 2019; Veta et al., 2019).”
- How Well Do Self-Supervised Models Transfer? (2020, background): “It consists of four datasets that exhibit increasing dissimilarity to natural images, CropDiseases [40], EuroSAT [24], ISIC2018 [54, 10] and ChestX [56].”
- Transformers in Medical Imaging: A Survey (2022, methodology): “Extensive experiments on ISIC 2016+PH2 [99], [100] and ISIC 2018 [101] validate the efficacy of their boundary-wise prior, as shown in Fig.”
- Comparison of Vision Transformers and Convolutional Neural Networks in Medical Image Analysis: A Systematic Review (2024, neutral): “Using the ISIC 2018 dataset [96], different architectures, including the U-Net, V-Net, Attention U-Net, TransUNet, and Swin-Unet models, were examined, and their performance in accurately segmenting skin lesions was evaluated.”
- Detection of Skin Cancer Based on Skin Lesion Images Using Deep Learning (2022, methodology): “In recent years, various deep learning algorithms have been applied to classify skin cancer, as outlined in Table 1, as well as other existing studies such as [30,31].”
- Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “Here, ISIC [14] is the in-domain data set, on which we train and also evaluate.”
- U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications (2020, background): “wl 2018 [273]. • ISIC archive: Database of Dermoscopy images from past and ongoing ISIC challenges. https://www.isic-archive.com/ Figure 15. Examples of Dermoscopy images from the ISIC 2018 challenge [274], [275]. • SICAS Medical Image Repository: Archive for MICCAI Brain Tumor Segmentation Challeng”
- Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “[38] 2D X-Ray, Dermoscopic ISIC Skin [39], Chexpert [40] Neighbor matching to generate pseudo-labels on a weight basis according to the embedding similarity with neighboring labeled data offline”
- Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, neutral): “ISIC Skin Cancer Data Set [4, 51] 21k 2 650× 450× 3 ✗ ✓ ✗ 1”
- No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems (2020, background): “ISIC The ISIC skin cancer dataset [12] is a public real-world dataset for classifying skin lesions as “malignant” or “benign.”
- VM-UNet: Vision Mamba UNet for Medical Image Segmentation (2024, neutral): “Meanwhile, SSM-CNN hybrid models (e.g., U-Mamba, SegMamba) retain CNN-dependent decoders, which limits the full exploitation of SSMs’ long-range modeling capabilities and may lead to suboptimal fusion of convolutional and SSM features.”
- BCN20000: Dermoscopic Lesions in the Wild (2019, background): “A significant player in the adoption of these algorithms in the community has been the International Skin Imaging Collaboration (ISIC), which has been organizing yearly challenges since 2016, where participants are asked to develop computer vision algorithms to segment and classify skin lesions in d”