Toward Objective Evaluation of Image Segmentation Algorithms

By Ranjith Unnikrishnan, Caroline Pantofaru, Martial Hebert. 2007

873 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Evaluation & Benchmarks

Citations in the corpus

Finely-grained annotated datasets for image-based plant phenotyping (2016, background): “Evaluation criteria: Several segmentation criteria are available for comparing between ground truth and algorithmic outcomes (Mezaris et al., 2003; Unnikrishnan et al., 2007).”
Shared Segmentation of Natural Scenes Using Dependent Pitman-Yor Processes (2008, methodology): “To quantitatively evaluate results, we measure overlap with held-out human segments via the Rand index [30].”
Significantly Fast and Robust Fuzzy C-Means Clustering Algorithm Based on Morphological Reconstruction and Membership Filtering (2018, methodology): “Inthis paper, four performance measures: probabilistic rand index (PRI) [42], the boundary displacement error (BDE) [43], the covering (CV) [41], and the variation of information (VI) [41] are used to quantitatively evaluate segmentations obtained by different algorithms against the Ground Truth…”
Segmentation using superpixels: A bipartite graph partitioning approach (2012, methodology): “We see that SAS ranks first in PRI and BDE by a large margin compared to previous methods, and second in VoI and third in GCE with performance quite close to the best one.”
Constrained parametric min-cuts for automatic object segmentation (2010, methodology): “The complexity of the CPMC procedure is O(kN 2 log(N )) for k seeds and image graphs with N nodes.”
Lesion border detection in dermoscopy images (2009, methodology): “Since the formulation of this measure is involved, the interested reader is referred to [24] and [25].”
Single-view reconstruction via joint analysis of image and shape collections (2015, methodology): “Accuracy was evaluated using the Rand index [Unnikrishnan et al. 2007].”
Comparison of segmentation algorithms for fluorescence microscopy images of cells (2011, methodology): “Some examples of quantitative algorithm comparison include use of the Normalized Probabilistic Rand (NPR) index (38) and methods used in research on face recognition.”
Maximin affinity learning of image segmentation (2009, methodology): “By using the simple graph partitioning algorithm of finding the connected components of the thresholded affinity graph, we are able to train an affinity classifier to directly minimize the Rand index of segmentations resulting from the graph partitioning.”
Learning Separable Filters (2013, methodology): “To avoid relying on a single one, which could be misleading, we considered multiple ones instead: Area Under Curve (AUC) computed on ROC curves, the F-measure, the Variation of Information (VI) [36], and the Rand Index (RI) [37].”
Accuracy Assessment Measures for Object-based Image Segmentation Goodness (2010, methodology): “The metrics described in Delves et al. (1992), Prieto and Allen (2003), and Unnikrishnan et al. (2007) made use of pixel information or post-classification labeling and were therefore not chosen for this re-implementation and comparison.”
Contour Detection and Hierarchical Image Segmentation (2011, methodology): “Noteworthy examples include the Probabilistic Rand Index, introduced in this context by [5], the Variation of Information [6], [7], and the Segmentation Covering criterion used in the PASCAL challenge [8].”
From contours to regions: An empirical evaluation (2009, background): “In [45], this drawback is addressed by normalization with an empirical estimation of its expected value.”
Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation (2015, background): “It has been deﬁned to deal with multiple ground-truth segmentations [159]: where S is the k -th human-labeled ground-truth, c ij is the event that pixels i and j have the same label and p ij is the corresponding probability.”
Crowdsourcing the creation of image segmentation algorithms for connectomics (2015, methodology): “Rand error has been proposed as a metric of segmentation performance (Unnikrishnan et al., 2007; Arbelaez et al., 2011), and has also been used as an objective function for directly optimizing the performance of machine learning algorithms (Turaga et al., 2009).”
CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts (2012, background): “The trend has gained momentum during the past 10 years, propelled by the creation of annotated benchmarks [7], [23] and new segmentation performance metrics [7], [24].”