CHAOS Challenge - Combined (CT-MR) Healthy Abdominal Organ Segmentation

By A. E. Kavur, N. Gezer, M. Baris, Pierre-Henri Conze, V. Groza, D. Pham, S. Chatterjee, Philipp Ernst, Savas Özkan, Bora Baydar and 17 more. Medical Image Anal., 2020

882 citations recorded by the source corpus.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/6d38bdd172cdbccb11745f5f031f848679117f25

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem? (2020, methodology): “Combined Healthy Abdominal Organ Segmentation (CHAOS)(15) [87] Liver 20/20 1 ISBI 2019”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, methodology): “[102] 3D CT, MRI BTCV [115], CHAOS [116] Different modalities”
Annotation-efficient deep learning for automatic medical image segmentation (2020, background): “%] as reported in the summary paper of the CHAOS challenge 33 ).”
Automated Design of Deep Learning Methods for Biomedical Image Segmentation (2019, methodology): “To demonstrate this, and also to support some of the core design choices made in nnU-Net, we systematically tested the performance of common pipeline variations in the nnU-Net blueprint parameters on 10 different datasets (Figure 6): the application of two alternative loss functions (Cross-entropy a”
AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation (2022, methodology): “With the development of related datasets [1, 11, 12, 5, 15], significant progress has been made in this area.”
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering (2023, neutral): “Modality Source Images QA pairs VQA-RAD [18] Radiology MedPix ® database 0.3k 3.5k PathVQA [12] Pathology PEIR Digital Library [14] 5k 32.8k SLAKE [23] Radiology MSD [3], ChestX-ray8 [36], CHAOS [15] 0.7k 14k VQA-Med-2021 [5] Radiology MedPix ® database 5k 5k PMC-VQA Mixture * PubMed Central ® 149k ”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “We evaluate on unseen MRI data [46] for liver segmentation task.”