Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

By A. Carass, Snehashis Roy, Amod Jog, Jennifer L. Cuzzocreo, E. Sweeney, A. Gherman, J. Button, James Nguyen, F. Prados, C. Sudre and 12 more. NeuroImage, 2017

383 citations recorded by the source corpus.

Abstract In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time‐points, and test data of fourteen subjects with a mean of 4.4 time‐points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state‐of‐the‐art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters. HighlightsPublic lesion data base of 21 training data sets and 61 testing data sets.Fully automated evaluation website.Comparison between 14 state‐of‐the‐art algorithms and 2 manual delineators.

Subfields: Biomedical Engineering, Pathology and Forensic Medicine, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/e9df6d7fa6b380c5db0ad4f29b9c32219aca05fe

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Pathology and Forensic Medicine

Sandra Vukusic

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Diffusion models in medical imaging: A comprehensive survey (2022, background): “[123] introduced in their work, which investigates three datasets (one private and two publicly available ones) with different modalities [127, 128].”
Automated Design of Deep Learning Methods for Biomedical Image Segmentation (2019, background): “The loss function is the sum of cross-entropy and Dice loss [4].”
Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study (2020, methodology): “MSSEG2015—The third MRI dataset in our experiments is the publicly available training set of the 2015 Longitudinal MS lesion segmentation challenge [19], which contains 21”
Domain Adaptation for Medical Image Analysis: A Survey (2021, neutral): “[47] propose a feature transformation-based DA method for microscopy image analysis.”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, background): “[73] 3D MRI Longitudinal Multiple Sclerosis Lesion Segmentation [91], ISLES 2015 [92], BraTS 2018 [23] fm Multi-scale features”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, methodology): “Following the latest recommendations in the literature on lesion detection applications [161], [162], [163], our main evaluation criterion was lesion-count F1 score;”
Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation (2018, background): “Recently, DL frameworks have been shown to outperform other segmentation methods on a variety of public challenge datasets [3,11,16], particularly on metrics focused on large pathologies.”
Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure (2018, background): “The second major challenge on MS lesion segmentation was held in 2015 at the IEEE ISBI international conference [8].”