SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks

By Xun Huang, Chengyao Shen, Xavier Boix, Qi Zhao. IEEE International Conference on Computer Vision, 2015

605 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Sensory Systems

View Paper: https://www.semanticscholar.org/paper/1281e443d2cf1c1dd71ed3b7b0376d408d0958af

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Sensory Systems

Citations in the corpus

StarMap for Category-Agnostic Keypoint and Viewpoint Estimation (2018, methodology): “Also, the heatmap representation has been used in saliency detection [8] and visual attention [45], which detects a region of image which is “important” in the context.”
Gaze Prediction in Dynamic 360° Immersive Videos (2018, background): “Recently, advances in deep learning have produced more accurate models [20] [33] [28] [37] [24].”
DeepGaze II: Reading fixations from deep features trained on object recognition (2016, methodology): “In contrast to DeepGaze I, which uses AlexNet, the SALICON model (Huang et al. 2015), DeepFix (Kruthiventi et al. 2015) and PDP (Jetley et al. 2016) use the better-performing VGG-19 network (Si-monyan and Zisserman 2014), whose features are retrained on saliency prediction using the SALICON dataset…”
Saliency in VR: How Do People Explore Virtual Environments? (2016, background): “Recently, advances in machine learning and particularly convolutional neuronal networks (CNNs) have fostered the convergence of top-down and bottom-up features for saliency prediction, producing more accurate models [20,34,42,59,63].”
Revisiting Video Saliency Prediction in the Deep Learning Era (2021, methodology): “For example, DeepFix [54], DeepNet [30] and SALICON net [28] ﬁne-tune VGG-16 [57] pre-trained on image classiﬁcation task.”
Learning Unsupervised Video Object Segmentation Through Visual Attention (2019, methodology): “Quantitative and qualitative results: We compare our DVAPmodule with 12 state-of-the-art visual attention models, including 5 deep models [75, 35, 73, 55, 28] and 7 traditional models [15, 20, 26, 61, 22, 32].”
A deep multi-level network for saliency prediction (2016, background): “The recent publication of the large dataset SALICON [12], collected thanks to crowdsourcing techniques, allows researches to increase the number of convolutional layers reducing the overfitting risk [13], [14].”
Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention (2017, methodology): “[16] compared three standard CNNs by applying them at two dierent image scales.”
Attentional Pooling for Action Recognition (2017, result): “Our initial experiments with existing saliency-based methods [21] were not promising.”
Deflecting Adversarial Attacks with Pixel Deflection (2018, methodology): “We prefer to use weakly supervised localization over saliency maps [17], as saliency maps are trained on human eye fixations and thus do not always capture object classes [25].”
Predicting the Driver's Focus of Attention: The DR(eye)VE Project (2017, background): “The recent success of deep networks has smoothed the aforementioned distinction, as models have become more and more powerful by seamlessly combining the beneﬁts of both top-down and bottom-up approaches, achieving state-of-the-art results on public benchmarks [12], [13], [27], [33], [34].”
Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model (2016, background): “Huang et al. [13] introduced an architecture consisting of a deep neural network applied at two different image scales.”
Deep Visual Attention Prediction (2017, methodology): “Training Deep Learning DeeFix [44] (no information) Yes Yes SALICON [45] max { w,h } =800 Yes Yes Mr-CNN [1] 400 × 400 Yes Yes SalNet [15] 320 × 240 Yes Yes Deep Gaze I [42] full size Yes Yes Saliency Uniﬁed [16] 417 × 417 Yes Yes eDN [13] 512 × 384 Yes Yes AIM [56] full size Yes No Judd Model [24]…”
Foveation-based Mechanisms Alleviate Adversarial Examples (2015, methodology): “It can be used for a fair comparison between Saliency Crop MP with 10 Crop MP, and to evaluate how much improvement comes from averaging multiple foveations.”
Faster gaze prediction with dense networks and Fisher pruning (2018, background): “…computational complexity and tried to ﬁnd the architecture which optimally optimizes a given Model AUC Center Bias 78% eDN [30] 82% SalNet [26] 83% DeepGaze I [19] 84% SAM-ResNet [5] 87% DSCLRCN 87% DeepFix [18] 87% SALICON [12] 87% DeepGaze II [20] 88 % FastGaze 85% DenseGaze 86% Table 1.”
SalGAN: Visual Saliency Prediction with Generative Adversarial Networks (2017, background): “[9] lies on minimizing loss functions based on metrics that are differentiable, such as NSS, CC, SIM and KL divergence to train the network (see [26] and [20] for the definition of these metrics.”