Deep Visual Attention Prediction

By Wenguan Wang, Jianbing Shen. 2017

639 citations recorded by the source corpus.

In this paper, we aim to predict human eye fixation with view-free scenes based on an end-to-end deep learning architecture. Although convolutional neural networks (CNNs) have made substantial improvement on human attention prediction, it is still needed to improve the CNN-based attention models by efficiently leveraging multi-scale features. Our visual attention network is proposed to capture hierarchical saliency information from deep, coarse layers with global saliency information to shallow, fine layers with local saliency response. Our model is based on a skip-layer network structure, which predicts human attention from multiple convolutional layers with various reception fields. Final saliency prediction is achieved via the cooperation of those global and local predictions. Our model is learned in a deep supervision manner, where supervision is directly fed into multi-level layers, instead of previous approaches of providing supervision only at the output layer and propagating this supervision back to earlier layers. Our model thus incorporates multi-level saliency predictions within a single network, which significantly decreases the redundancy of previous approaches of learning multiple network streams with different input scales. Extensive experimental analysis on various challenging benchmark data sets demonstrate our method yields the state-of-the-art performance with competitive inference time.11Our source code is available at https://github.com/wenguanwang/deepattention.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Why Should We Add Early Exits to Neural Networks? (2020, background): “en successfully applied to, among others, face recognition [66], object detection [15], image segmentation [16], pose estimation [27],saliencydetection[43],metriclearning[76],visualattentionprediction[74],andsuper resolution [36,70]. JT approaches are also popular in the case of recurrent neural net”
Revisiting Video Saliency Prediction in the Deep Learning Era (2021, methodology): “ND-avg represents the average results of three top-performing non-deep learning models: GBVS [27], Fang et al. [14] and ITTI [26]; D-avg indicates average results of three top-performing deep learning models: ACLNet, SalGAN [31] and DVA [29], according to Table 8.”
Probabilistic Knowledge Transfer for Lightweight Deep Representation Learning (2020, methodology): “To this end, reinforcement learning [46] and/or attention mechanisms [47] can be employed, allowing to better select and/or pair the layers between the student and teacher models, further improving the efﬁciency of KT. Maria Tzelepi received the B.Sc. degree in informatics and the M.Sc. degree in…”
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model (2018, methodology): “OM-CNN, Two-stream, SALICON, DVA, Shallow-Net, and Deep-Net are deep learning models, and others are classical saliency models.”
Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception (2020, methodology): “By denoting Ŝ, Ê, Ŷ = G(X) as the generated output, we formulate the loss terms as follows: 1) Information Loss for saliency prediction is measured by a standard cross-entropy function [51, 64].”
See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks (2019, background): “Differentiable attentions, which are inspired by human perception [13, 58], have been widely studied in deep neural networks [26, 56, 38, 23, 57, 62, 15].”
DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer (2023, background): “Because of promising feature representation capability, DL-based studies have attracted increasing attention in the computer vision (CV) community [15], [16], [17], [18], [19], [20], [21].”
Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, methodology): “Inspired by the human attention mechanism, substantial algorithms have been developed over the last few decades [51]–[53].”
RGB-D salient object detection: A survey (2020, background): “ould help with feature or saliency map fusion and further boost the SOD performance. Attention-induced Fusion. Attention mechanisms have been widely applied to various deep learning-based tasks [178]–[181], allowing networks to selectively pay attention to a subset of regions for extracting discrimi”
A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping (2019, background): “Deep learning-based attention models [29], [30], [31], [32], [33], [34] become increasingly popular in recent years, driven by the success of deep learning in object recognition and large-scale visual attention dataset (e.”
Pyramid Dilated Deeper ConvLSTM for Video Salient Object Detection (2018, background): “, eye fixation prediction [41, 39] and salient object detection [49, 47].”
Learning Unsupervised Video Object Segmentation Through Visual Attention (2019, methodology): “Recently, deep learning based attention models [71, 28, 55, 73, 75] were proposed and generally yield better performance.”
Camouflaged Object Detection (2020, background): “…similarities between the target object and the background make COD far more challenging than the traditional salient object detection [1,5,17,25,62– 66, In addition to its scientiﬁc value, COD is also beneﬁcial for applications in the ﬁelds of computer vision (for search-and-rescue work, or…”
Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency (2019, methodology): “, DVA [8], SalNet [7] and SSNet [9]) or temporal saliency (e.”