The Liver Tumor Segmentation Benchmark (LiTS)

By Patrick Bilic, P. Christ, Eugene Vorontsov, G. Chlebus, Hao Chen, Qi-Yan Dou, Chi-Wing Fu, Xiao Han, P. Heng, Jürgen W. Hesser and 50 more. Medical Image Anal., 2019

1,500 citations recorded by the source corpus.

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper−/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/0655dcaa39cf41a3609974840f91300d73b4aed1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

How Artificial Intelligence Is Shaping Medical Imaging Technology: A Survey of Innovations and Applications (2023, background): “In [8], the authors focus on the segmentation of liver lesions in CT images of the LiTS dataset [94].”
Continual Self-Supervised Learning: Towards Universal Multi-Modal Medical Data Representation Learning (2023, background): “…infected region segmentation (QaTa-COV19-v2) [10]); (3) CT: COVID-19 classification (RICORD [43]) and liver and liver tumor segmentation (LiTS [3]); (4) MRI: vestibular schwannoma segmentation (VS [37]) and left atrium segmentation (LA [47]); (5) Pathological imaging: colorectal cancer and…”
CHAOS Challenge - Combined (CT-MR) Healthy Abdominal Organ Segmentation (2020, background): “LiTS - Liver Tumor Segmentation Challenge (Bilic et al., 2019) is another”
Transformation-Consistent Self-Ensembling Model for Semisupervised Medical Image Segmentation (2019, methodology): “CT dataset, Dice per case score is employed to measure the accuracy of the liver segmentation result, according to the evaluation of the 2017 LiTS challenge [57].”
INet: Convolutional Networks for Biomedical Image Segmentation (2021, methodology): “…of the proposed method in the segmentation of biomedical images, we carried out experiments on nine public imaging datasets: three brain (BraTS2013, LMS, BTD) one heart MRI [42]–[44], one liver one spleen CT [44], [45], one lung X-ray [9], one colon endoscopic [46], and one nerve ultrasonic [47].”
Segment Anything in Medical Images (2023, background): “, T1, T2, ADC, FLAIR) [24][10][6][41][5][8][29][42], abdomen tumor, COVID-19 infection, gallbladder, head and neck tumor, liver, pancreas, pleural effusion, and stomach segmentation in CT images [34][19][7][33][32][42][4][28][27][12], breast tumor, liver, and vessel segmentation in ultrasound images”
AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem? (2020, methodology): “NIH Pancreas(9) [21], [22], [23] Pancreas 80 1 The Cancer Imaging Archive 2015 VISCERAL Anatomy Benchmark(10) [86] 20 anatomical structures 80/40 1 ISBI and ECIR 2015 Liver Tumor Segmentation Benchmark (LiTS)(11) [16] Liver and tumor 131/70 7 ISBI and MICCAI 2017 Medical Segmentation Decathlon (MSD)”
Box-supervised Instance Segmentation with Level Set Evolution (2022, methodology): “To evaluate our proposed approach, we conduct experiments on four challenging datasets, including Pascal VOC [11] and COCO [30], remote sensing dataset iSAID [50] and medical dataset LiTS [3].”
The Medical Segmentation Decathlon (2021, methodology): “The data was acquired in the IRCAD Hˆopitaux Universitaires, Strasbourg, France and contained a subset of patients from the 2017 Liver Tumor Segmentation (LiTS) challenge [16].”
Medical Image Segmentation Using Deep Learning: A Survey (2020, neutral): “Thus there are many datasets on the brain and abdomen such as BRATS [3] [153] [154], ISLES [155], KITS [156], LITS [157], CHAOS [158], etc.”
Federated learning for medical image analysis: A survey (2023, background): “(2023) [137] Tumor segmentation LiTS [138] CT U-Net Wicaksana et al.”
The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge (2019, background): “Despite these challenges, recent work in this area has demonstrated impressive performance on the 3D segmentation of a wide variety of anatomical structures and lesions in cross-sectional imaging (Maier et al., 2017; Bakas et al., 2018; Bilic et al., 2019; Zhuang et al., 2019).”
Segment Anything Model for medical image analysis: an experimental study (2023, background): “CT-Liver The Liver Tumor Segmentation Benchmark (LiTS) [4] CT 1 Liver 5,501”
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection (2023, background): “The CT scans in LiTS [3], KiTS [24], and MSD Pancreas [1] contain tumors in liver, kidney and pancreas, respectively.”
MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis (2020, methodology): “OragnMNIST Coronal [15, 16] Abdominal CT Multi-Class (11) 13,000 2,392 8,268”
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation (2022, methodology): “[109] 3D CT NIH Pancreas [28], LiTS dataset [37] Different transformations”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, methodology): “Ours uses additional 70 volumes from LITS2017 [9] testing set as unlabeled data samples.”