Visual Saliency Detection Based on Multiscale Deep CNN Features

By Guanbin Li, Yizhou Yu. IEEE Transactions on Image Processing, 2016

363 citations recorded by the source corpus.

Visual saliency is a fundamental problem in both cognitive and computational sciences, including computer vision. In this paper, we discover that a high-quality visual saliency model can be learned from multiscale features extracted using deep convolutional neural networks (CNNs), which have had many successes in visual recognition tasks. For learning such saliency models, we introduce a neural network architecture, which has fully connected layers on top of CNNs responsible for feature extraction at three different scales. The penultimate layer of our neural network has been confirmed to be a discriminative high-level feature vector for saliency detection, which we call deep contrast feature. To generate a more robust feature, we integrate handcrafted low-level features with our deep contrast feature. To promote further research and evaluation of visual saliency models, we also construct a new large database of 4447 challenging images and their pixelwise saliency annotations. Experimental results demonstrate that our proposed method is capable of achieving the state-of-the-art performance on all public benchmarks, improving the F-measure by 6.12% and 10%, respectively, on the DUT-OMRON data set and our new data set (HKU-IS), and lowering the mean absolute error by 9% and 35.3%, respectively, on these two data sets.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/db029abd3316af48bd97a6552c586f2913631748

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Huchuan Lu
2. Deng-Ping Fan
3. Jianbing Shen
4. Huan Fu
5. Ming-Ming Cheng
6. Qingming Huang
7. Yizhou Yu
8. Liang Lin

Citations in the corpus

• Object Detection With Deep Learning: A Review (2018, methodology): “The following approaches are evaluated: CHM [150], RC [151], DRFI [152], MC [138], MDF [146], LEGS [136], DSR”
• Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation (2018, background): “and combine the final output map in the last layer [32], [33], our M-Net employs the average pooling layer to downsample the image naturally and construct a multi-scale input in the encoder path.”
• Context-Aware Semantic Inpainting (2017, background): “FCNs have achieved excellent results on semantic segmentation [27], [28]; saliency detection [29]–[32]; automatic image colorization [33]; as well other pixel-wise inferring-based”
• BASNet: Boundary-Aware Salient Object Detection (2019, methodology): “We compare our method with 15 state-of-the-art models, PiCANetR [39], BMPM [72], R3Net [6], PAGRN [76], RADF [19], DGRL [65], RAS [4], C2S [36], LFR [73], DSS [17], NLDF [41], SRM [64], Amulet [74], UCF [75], MDF [35].”
• Reverse Attention for Salient Object Detection (2018, methodology): “We comprehensively evaluated our method on six representative datasets, including MSRA-B [36], HKU-IS [37], ECSSD [38], PASCAL-S [39], SOD [40], and DUT-OMRON [41], which contain 5000, 4447, 1000, 850, 300, 5168 well annotated images, respectively.”
• U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection (2020, methodology): “The curves are consistent with the Table 3 and 4 which demonstrate the state-of-the-art performance of our U 2 -Net on DUT-OMRON, HKU-IS and ECSSD and competitive performance on other datasets.”