Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation

By Simon Keith Warfield, Kelly H. Zou, William M. Wells. IEEE Transactions on Medical Imaging, 2004

2,250 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/d860504ff0a3faaeeccf72e9a6c982b644ed16a2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging (2013, methodology): “STAPLE (Commowick and Warfield, 2009; Warfield et al., 2004) provides a framework to compare the accuracy of multiple experts and create a silver standard.”
Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge (2013, methodology): “It is well known that combining the results of multiple human observers often leads to a better segmentation than using the segmentation of only a single observer (Warﬁeld et al., 2004).”
A review of atlas-based segmentation for magnetic resonance brain images (2011, methodology): “This approach used a set of topological atlases to define healthy tissue samples, obtained using the STAPLE algorithm [74], and to create a probabilistic atlas from the average of these manual segmentations.”
DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy (2017, methodology): “In addition, we compare to spatial STAPLE (Asman and Landman, 2012), which is an extension of the popular simultaneous truth and performance level estimation (STAPLE) method (Warfield et al., 2004).”
Advanced Level-Set-Based Cell Tracking in Time-Lapse Fluorescence Microscopy (2010, methodology): “A reference segmentation for each image was calculated from the manual segmentations, using the "simultaneous truth and performance level estimation (STAPLE) algorithm [ 59 ].”
Multi-Atlas Segmentation of Biomedical Images: A Survey (2014, methodology): “, 2007; Crum, 2009), the segmentation of tumors (Zikic et al., 2014a; Wang and Yushkevich, 2013b; Warfield et al., 2004), eyes and optic nerves (Datteri et al.”
QIN “Radiomics: The Process and the Challenges” (2012, methodology): “Many metrics have been used, like volume, center of volume and maximum surface distance, to compare characteristics like robustness and accuracy [57,58].”
Computer Methods and Programs in Biomedicine (neutral): “To solve this issue, the Simultaneous Truth And Performance Level Estimation (STAPLE) has been introduced in [127] .”
Learning From Crowds (2010, background): “(2009) address the issue of active learning in this scenario—How to jointly learn the accuracy of labeling sources and obtain the most informative labels for the active learning task? There has also been some work in the medical imaging community (Warfield et al., 2004; Cholleti et al., 2008).”
Standardized Evaluation Methodology and Reference Database for Evaluating Coronary Artery Centerline Extraction Algorithms (2009, background): “Moreover, in order to get a good reference standard, annotations are needed from multiple observers and combining annotations from multiple observers is known to be difficult (Warfield et al., 2004) and until recently unexplored for three-dimensional curves (Walsum et al.”
Histology image analysis for carcinoma detection and grading (2012, methodology): “Can the existing methods to estimate ground truth that have been established for radiology images, e.g. STAPLE [150], be applied to histological applications.”
Deep learning with noisy labels: exploring techniques and remedies in medical image analysis (2019, methodology): “• STAPLE + iMAE loss.”
MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer (2023, methodology): “Then we use STAPLE algorithm(Warfield, Zou, and Wells 2004) to fuse the different samples.”
Preparing Medical Imaging Data for Machine Learning (2020, methodology): “Segmentation performance can be evaluated by using either the Dice coefficient or the more advanced simultaneous truth and performance level estimation, or STAPLE, algorithm (44).”
MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model (2022, methodology): “The segmentation performance was evaluated using Dice score and IoU for 2D images, and an additional 95th percentile Hausdorff Distance (HD95) metric for 3D volumes.”
Automatic anatomical brain MRI segmentation combining label propagation and decision fusion (2006, methodology): “A refinement of the method presented by Warfield et al. (2004) weights the classifiers based on an expectation–maximization algorithm ( " STAPLE " ).”