Structure-Measure: A New Way to Evaluate Foreground Maps

By Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, Ali Borji. International Journal of Computer Vision, 2017

1,869 citations recorded by the source corpus.

Foreground map evaluation is crucial for gauging the progress of object segmentation algorithms, in particular in the field of salient object detection where the purpose is to accurately detect and segment the most salient object in a scene. Several measures (e.g., area-under-the-curve, F1-measure, average precision, etc.) have been used to evaluate the similarity between a foreground map and a ground-truth map. The existing measures are based on pixel-wise errors and often ignore the structural similarities. Behavioral vision studies, however, have shown that the human visual system is highly sensitive to structures in scenes. Here, we propose a novel, efficient (0.005 s per image), and easy to calculate measure known as S-measure (structural measure) to evaluate foreground maps. Our new measure simultaneously evaluates region-aware and object-aware structural similarity between a foreground map and a ground-truth map. We demonstrate superiority of our measure over existing ones using 4 meta-measures on 5 widely-used benchmark datasets. Furthermore, we conduct a behavioral judgment study over a new database. Data from 45 subjects shows that on average they preferred the saliency maps chosen by our measure over the saliency maps chosen by the state-of-the-art measures. Our experimental results offer new insights into foreground map evaluation where current measures fail to truly examine the strengths and weaknesses of models. Code: https://github.com/DengPingFan/S-measure.

Subfields: Computer Vision and Pattern Recognition, Sensory Systems

View Paper: https://www.semanticscholar.org/paper/381d56f925530bc0fcd11e7cd4c4f7bb033fe000

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Ling Shao
  2. Luc Van Gool
  3. Ming-Ming Cheng
  4. Deng-Ping Fan
  5. Ming-Hsuan Yang
  6. Dacheng Tao
  7. Jianbing Shen
  8. Huchuan Lu

In Sensory Systems

  1. Jianbing Shen
  2. Wenguan Wang
  3. Ming-Ming Cheng
  4. Ling Shao
  5. Deng-Ping Fan
  6. Huchuan Lu
  7. Junwei Han
  8. Huan Fu

Citations in the corpus