Enhanced-alignment Measure for Binary Foreground Map Evaluation

By Deng-Ping Fan, Cheng Gong, Yang Cao, Bo Ren, Ming-Ming Cheng, Ali Borji. International Joint Conference on Artificial Intelligence, 2018

1,626 citations recorded by the source corpus.

The existing binary foreground map (FM) measures address various types of errors in either pixel-wise or structural ways. These measures consider pixel-level match or image-level information independently, while cognitive vision studies have shown that human vision is highly sensitive to both global information and local details in scenes. In this paper, we take a detailed look at current binary FM evaluation measures and propose a novel and effective E-measure (Enhanced-alignment measure). Our measure combines local pixel values with the image-level mean value in one term, jointly capturing image-level statistics and local pixel matching information. We demonstrate the superiority of our measure over the available measures on 4 popular datasets via 5 meta-measures, including ranking models for applications, demoting generic, random Gaussian noise maps, ground-truth switch, as well as human judgments. We find large improvements in almost all the meta-measures. For instance, in terms of application ranking, we observe improvement ranging from 9.08% to 19.65% compared with other popular measures.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d1e5942d8cb764e58f4c38a0c254ca56782a98c3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CaraNet: context axial reverse attention network for segmentation of small medical objects (2021, methodology): “In addition to mean Dice and mean IoU, we also select other four measurement metrics: weighted dice metric Fβ w , MAE, enhanced alignment metric Εφ max [17] and structural”
Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network (2019, background): “In virtue of recent development of object detection [41, 30, 11, 9, 5, 8, 56] and segmentation [1, 55, 33, 53, 6, 7] based on CNN [15, 20, 47, 25], scene text detection has witnessed great progress [46, 42, 57, 31, 35, 49, 24].”
Label Decoupling Framework for Salient Object Detection (2020, methodology): “Mean F -measure (mF ), E-measure (Eξ) [9], weighted F -measure (F ω β ) and S-measure (Sα) are also widely used to evaluate saliency maps.”
Visual Saliency Transformer (2021, background): “Maximum enhanced-alignment measure Emax ξ [14] simultaneously considers pixel-level errors and image-level errors.”
Suppress and Balance: A Simple Gated Network for Salient Object Detection (2020, methodology): “We adopt several metrics widely used in RGB-D SOD for quantitative evaluation: F-measure score, mean absolute error (MAE, M), the recently released S-measure (Sm) [13] and E-measure (Em) [14] scores.”
Salient Object Detection in the Deep Learning Era: An In-Depth Survey (2019, background): “• Enhanced-alignment measure (E-measure) [129] considers global means of the image and local pixel matching simultaneously: where φ S is the enhanced alignment matrix, which reﬂects the correlation between S and G after subtracting their global means, respectively.”
Weakly-Supervised Salient Object Detection via Scribble Annotations (2020, methodology): “Evaluation Metrics: Four evaluation metrics are used, including Mean Absolute Error (MAE M), Mean F-measure (Fβ), mean E-measure (Eξ [7]) and our proposed saliency structure measure (Bμ).”
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders (2020, methodology): “Here, we adopt mean Fβ and mean Eξ [15].”
Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images (2020, background): “We also introduce three golden metrics from the object detection field, i.e., Structure Measure [62], Enhance-alignment Measure [63], and Mean Absolute Error.”
Shifting More Attention to Video Salient Object Detection (2019, background): “For ViSal dataset (the ﬁrst speciﬁcally-designed dataset for VSOD), their average performance ( e.g ., max E-measure [19], max F-measure, or S-measure) is even higher than 0.9.”
Concealed Object Detection (2021, background): “Finally, recent studies [71], [81] have suggested that the weighted Fmeasure (Fw β ) [82] can provide more reliable evaluation results than the traditional Fβ .”
Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection (2022, methodology): “For COD and SOD, we use seven common metrics for evaluation based on [32,33], including S-measure [6] (S m ), weighted F-measure [30] (F ωβ ), mean absolute error (MAE), F-measure [1] (F β ), E-measure [7] (E m ), precision-recall (PR) curve and F β -threshold curve (F β curve).”
F3Net: Fusion, Feedback and Focus for Salient Object Detection (2019, background): “5 ) (Fan et al. 2017) and E-measure ( E ξ ) (Fan et al. 2018) are also widely used to evaluate salient maps.”
PraNet: Parallel Reverse Attention Network for Polyp Segmentation (2020, methodology): “To evaluate pixel-level and global-level similarity, we adopt the recently released enhanced-alignment metric E maxφ [6].”
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (2021, methodology): “We employ six widely-used evaluation metrics, including Dice [85], IoU, mean absolute error (MAE), weighted Fmeasure (F β ) [86], S-measure (Sα) [87], and E-measure (Eξ) [88], [89] to evaluate the model performances.”
SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection (2022, background): “They are the precision-recall (PR) curve [83], S-measure [84], F-measure [85], E-measure [86] and mean absolute error (MAE) [87].”
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “E-measure is the recently proposed Enhanced alignment measure [59] from the binary map evaluation field.”
Cross-level Feature Aggregation Network for Polyp Segmentation (2023, background): “Additionally, four metrics are introduced, which are widely used in the ﬁeld of object detection [32,52,53] , including S-measure ( S α ) [54] , F-measure [55] ( F w β ), E φ [56] , and mean absolute error ( M ) [57] .”
RGB-D salient object detection: A survey (2020, background): “[159], and enhanced-alignment measure (E-measure) [160].”