Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model
By Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara. IEEE Transactions on Image Processing, 2016
608 citations recorded by the source corpus.
Data-driven saliency has recently gained a lot of attention thanks to the use of convolutional neural networks for predicting gaze fixations. In this paper, we go beyond standard approaches to saliency prediction, in which gaze maps are computed with a feed-forward network, and present a novel model which can predict accurate saliency maps by incorporating neural attentive mechanisms. The core of our solution is a convolutional long short-term memory that focuses on the most salient regions of the input image to iteratively refine the predicted saliency map. In addition, to tackle the center bias typical of human eye fixations, our model can learn a set of prior maps generated with Gaussian functions. We show, through an extensive evaluation, that the proposed architecture outperforms the current state-of-the-art on public saliency prediction datasets. We further study the contribution of each key component to demonstrate their robustness on different scenarios.
Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Human-Computer Interaction
View Paper: https://www.semanticscholar.org/paper/b7e336a7d3cd82ae8cd5e85d3cb62f0b6091a0e5
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
- Rita Cucchiara
- Lorenzo Baraldi
- Giuseppe Serra
- Marcella Cornia
- Ruohan Zhang
- Dana H. Ballard
- Scott Niekum
- Akanksha Saran
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Yi Yang
- Rita Cucchiara
- Trevor Darrell
- Jitendra Malik
In Human-Computer Interaction
- Ruohan Zhang
- Scott Niekum
- Akanksha Saran
- Dana H. Ballard
- Mary M. Hayhoe
- Sihang Guo
- Rita Cucchiara
- Lorenzo Baraldi
Citations in the corpus
- Saliency-based Sequential Image Attention with Multiset Prediction (2017, background): “However, the modular architecture allows for plugging in alternative, rigorously evaluated saliency models such as a state-of-the-art saliency model trained with human fixation data [10].”
- How Much Position Information Do Convolutional Neural Networks Encode? (2020, background): “, 2015) and saliency detection (Cornia et al., 2018; Li et al., 2014).”
- Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention (2017, methodology): “To compute saliency maps, we employ the approach in [8], which features state of the art results on popular saliency benchmarks, such as the MIT Saliency Benchmark [4] and the SALICON dataset [19].”
- Faster gaze prediction with dense networks and Fisher pruning (2018, background): “…computational complexity and tried to find the architecture which optimally optimizes a given Model AUC Center Bias 78% eDN [30] 82% SalNet [26] 83% DeepGaze I [19] 84% SAM-ResNet [5] 87% DSCLRCN 87% DeepFix [18] 87% SALICON [12] 87% DeepGaze II [20] 88 % FastGaze 85% DenseGaze 86% Table 1.”
- Human Gaze Assisted Artificial Intelligence: A Review (2020, background): “…(object referring) [Bala-jee Vasudevan et al. , 2018], generating descriptive captions for images [Sugano and Bulling, 2016; Tavakoli et al. , 2017; Cornia et al. , 2018a; Chen and Zhao, 2018; He et al. , 2019b] and videos [Yu et al. , 2017], as well as visual question answering [Qiao et al. ,…”
- Dual Attention Guided Gaze Target Detection in the Wild (2021, methodology): “For these reasons, gaze analysis has widely been used in neurophysiology studies [36, 10], relevant saliency prediction [8, 34], and social awareness tracking [7, 29, 30].”