What is a good evaluation measure for semantic segmentation?

By Gabriella Csurka, Diane Larlus, Florent Perronnin. 2013

513 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Knowledge Distillation for Incremental Learning in Semantic Segmentation (2019, methodology): “These benchmarks have been widely used to evaluate semantic segmentation schemes [19, 54].”
Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data (2018, methodology): “To quantitatively assess the performance for each class, we compare the binary predicted map and the binary ground-truth using two evaluation metrics: intersection over union (IoU) [50] and F1-score [3].”
DeeperLab: Single-Shot Image Parser (2019, background): “We refer the interested readers to [4, 89] for an overview of the related literature.”
Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification (2017, unknown): “We then add the intersection over union (IoU) criterion, an object-based overlap measure typically used for imbalanced data sets [38].”
Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization (2018, methodology): “The IoU, also known as the Jaccard similarity coeﬃcient, is the most commonly used in benchmarking (Csurka, Larlus, & Perronnin, 2013).”
The Lov´asz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (methodology): “Using the per-image Jaccard index is known to have better perceptual accuracy by reducing the bias towards large instances of the object classes in the dataset [6].”
Adaptive Weighting Multi-Field-Of-View CNN for Semantic Segmentation in Pathology (2019, background): “These metrics [14] are deﬁned as: where M is the number of classes, and TP c , FP c , and FN c are the numbers of true positives, false positives, and false negatives for class c , respectively.”
Zero-Shot Semantic Segmentation (2019, background): “We note that for semantic segmentation, the pixel accuracy metric (PA) is biased toward the more dominant classes and might suggest misleading conclusions [12], as opposed to IoU metrics.”
The Lovasz-Softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-Over-Union Measure in Neural Networks (2017, unknown): “the object classes in the dataset [6].”
Enhanced-alignment Measure for Binary Foreground Map Evaluation (2018, methodology): “Various evaluations based on F (cid:12) -measures [Csurka et al. , 2013; Margolin et al. , 2014; Shi et al. , 2015] and other measures ( e.g. [Movahedi and Elder, 2010; Villegas and Marichal, 2004; McGuinness and O’connor, 2010]) have been reported in the past.”
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation (2016, background): “In order to judge the quality of a segmentation, the choice of a suitable metric is largely dependent on the end goal of the final application [10].”
Probabilistic Object Detection: Definition and Evaluation (2018, background): “AP-based measures have biased the community to develop object detectors with high recall rate and localisation precision, but these measures have several weaknesses [3, 16, 36].”