Supervised Evaluation of Image Segmentation and Object Proposal Techniques

By Jordi Pont-Tuset, Ferran Marqués. 2016

145 citations recorded by the source corpus.

This paper tackles the supervised evaluation of image segmentation and object proposal algorithms. It surveys, structures, and deduplicates the measures used to compare both segmentation results and object proposals with a ground truth database; and proposes a new measure: the precision-recall for objects and parts. To compare the quality of these measures, eight state-of-the-art object proposal techniques are analyzed and two quantitative meta-measures involving nine state of the art segmentation methods are presented. The meta-measures consist in assuming some plausible hypotheses about the results and assessing how well each measure reflects these hypotheses. As a conclusion of the performed experiments, this paper proposes the tandem of precision-recall curves for boundaries and for objects-and-parts as the tool of choice for the supervised evaluation of image segmentation. We make the datasets and code of all the measures publicly available.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “011 for PASCAL-Context and NYUD, respectively, using the package of [48].”
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation (2015, methodology): “Precision-Recall curves for boundaries [35] (left) and for objects and parts [36] (right).”
What Do Different Evaluation Metrics Tell Us About Saliency Models? (2016, background): “model, which has more zero values, fares worse than the chance model. One possible meta-measure for selecting metrics for evaluation is how much better one baseline is over another (e.g., [23], [56], [65]). However, the optimal ranking of baselines is likely to be different across applications: in s”
Enhanced-alignment Measure for Binary Foreground Map Evaluation (2018, methodology): “We use 4 meta-measures proposed in [10, 24, 28], as well as a new one (Sec.”
Probabilistic Object Detection: Definition and Evaluation (2018, background): “AP-based measures have biased the community to develop object detectors with high recall rate and localisation precision, but these measures have several weaknesses [3, 16, 36].”
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation (2017, methodology): “6 Boundary Region Fb Fop SC PRI VI Input ODS OIS ODS OIS ODS OIS ODS OIS ODS OIS Si ng le -”
Piecewise Flat Embedding for Image Segmentation (2018, methodology): “Other methods are evaluated through the SEISM toolbox [64].”
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation (2016, background): “As justified in [37], for images one can use two complementary points of view, regionbased and contour-based measures.”
Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks (2016, methodology): “PASCAL VOC 2012 evaluation : Precision-recall curves for evaluation of boundaries ( F b [26]), and regions ( F op [27]).”
Decoupling Zero-Shot Semantic Segmentation (2021, background): “Class-Agnostic Segmentation is a long-standing problem and has been extensively studied in computer vision [3,4,14,44,50,52,64].”
Large-Scale Interactive Object Segmentation With Human Annotators (2019, result): “We observe a similar trend when comparing boundary quality (F-measure at 5 pixels [23, 29]): 75% for our corrective click masks versus 65% for COCO polygons, and 79% for the reference free-painted masks.”
Deep Retinal Image Understanding (2016, methodology): “As a summary measure (in brackets in the legend) we compute the Dice coefﬁcient or F1-measure (equivalent [16] to the Jaccard index) of the optimal point (marked in the lines).”