Shifting More Attention to Video Salient Object Detection

By Deng-Ping Fan, Wenguan Wang, Ming-Ming Cheng, Jianbing Shen. Computer Vision and Pattern Recognition, 2019

489 citations recorded by the source corpus.

The last decade has witnessed a growing interest in video salient object detection (VSOD). However, the research community long-term lacked a well-established VSOD dataset representative of real dynamic scenes with high-quality annotations. To address this issue, we elaborately collected a visual-attention-consistent Densely Annotated VSOD (DAVSOD) dataset, which contains 226 videos with 23,938 frames that cover diverse realistic-scenes, objects, instances and motions. With corresponding real human eye-fixation data, we obtain precise ground-truths. This is the first work that explicitly emphasizes the challenge of saliency shift, i.e., the video salient object(s) may dynamically change. To further contribute the community a complete benchmark, we systematically assess 17 representative VSOD algorithms over seven existing VSOD datasets and our DAVSOD with totally ~84K frames (largest-scale). Utilizing three famous metrics, we then present a comprehensive and insightful performance analysis. Furthermore, we propose a baseline model. It is equipped with a saliency shift- aware convLSTM, which can efficiently capture video saliency dynamics through learning human attention-shift behavior. Extensive experiments open up promising future directions for model development and comparison.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/21648dd098a3c0ae871844b254775fec14df6904

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Attentional Feature Fusion (2020, background): “The attention mechanism in deep learning, which mimics the human visual attention mechanism [5, 8], is originally developed on a global scale.”
Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection (2019, background): “It has a wide range of applications, including video/image segmentation [17, 40], object recognition [46], visual tracking [ 3], foreground maps evaluation [14, 15], image retrieval [6,16,22,38], content-aware image editing [8], information discovery [58], photo synthesis [5, 29], and weakly…”
EGNet: Edge Guidance Network for Salient Object Detection (2019, background): “Besides, there are many works focusing on video salient object detection [12, 54] and RGB-D salient object detection [11, 66].”
Multinomial Distribution Learning for Effective Neural Architecture Search (2019, unknown): “For the numbers of filters F and F , a common heuristic in most human designed convolutional neural networks [11, 14, 16, 27, 10, 31] is to double F whenever the spatial feature map is halved.”
Revisiting Video Saliency Prediction in the Deep Learning Era (2021, background): “A related topic is salient object detection [63], [64] that uniformly highlights salient object regions in images [65], [66], [67], [68], [69], videos [70], [71], [72], [73], image/video groups [74], or RGBD data [75], [76].”
Label Decoupling Framework for Salient Object Detection (2020, background): “Salient object detection (SOD) [1, 6, 10, 11, 12] aims at identifying the most visually attractive objects or parts in an image or video, which is widely applied as a pre-processing”
Salient Object Detection via Integrity Learning (2021, background): “We refer readers to recent benchmark work [51] for more details.”
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders (2020, unknown): “, VSOD [19], RGB SOD [13, 65], Co-SOD [17]).”
RGB-D salient object detection: A survey (2020, background): “SOD plays a key role in a range of real-world applications, such as stereo matching [11], image understanding [12], co-saliency detection [13], action recognition [14], video detection and segmentation [15]–[18], semantic segmentation [19], [20], medical image segmentation [21]–[23], object tracking”
Efficient and Accurate Arbitrary-Shaped Text Detection With Pixel Aggregation Network (2019, background): “In virtue of recent development of object detection [41, 30, 11, 9, 5, 8, 56] and segmentation [1, 55, 33, 53, 6, 7] based on CNN [15, 20, 47, 25], scene text detection has witnessed great progress [46, 42, 57, 31, 35, 49, 24].”
Stacked Cross Refinement Network for Edge-Aware Salient Object Detection (2019, background): “Salient object detection [1, 5, 10, 11, 17] aims to detect and segment the most attractive objects in images or videos.”
A Survey of Deep Learning-Based Object Detection (2019, background): “Video salient object detection datasets [237]–[243] provide benchmarks for video salient object detection, and exist-”
Concealed Object Detection (2021, background): “Similar to in RGB-D SOD [53], RGB-T SOD [100], CoSOD [101], [102], and VSOD [103], these modalities can be audio, thermal, group image, or depth data, raising new challenges under speciﬁc scenes.”
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection (2019, background): “Recently, this fundamental task plays an important role in various computer vision applications [15,21,29,37], e.”
A Survey on Deep Learning Technique for Video Segmentation (2021, background): “Core Architecture InstanceFlow Training Dataset 2017 FSEG [66] CVPR Two-Stream FCN Object ✓ ImageNet VID [67] + DAVIS16 [17] SFL [68] ICCV Two-Stream FCN Object ✓ DAVIS16 [17] LVO [69] ICCV Two-Stream FCN Object ✓ DAVIS16 [17] LMP [70] ICCV FCN Object ✓ FT3D [71] NRF [72] ICCV FCN Object ✓ Youtube-O”
Saliency Detection via a Multiple Self-Weighted Graph-Based Manifold Ranking (2020, background): “The goal of saliency detection is to obtain the most compelling regions from images [2], [3] or videos [4].”
JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (2020, background): “It has many useful applications, including object segmentation and recognition [27, 32, 39, 51, 70, 79], image/video compression [24], video detection/summarization [19, 41], content-based image editing [14, 23, 42, 57, 63], informative common object discovery [71, 72], and image retrieval [8, 22, 3”