Saliency-based Sequential Image Attention with Multiset Prediction
By Sean Welleck, Jialin Mao, Kyunghyun Cho, Zheng Zhang. 2017
23 citations recorded by the source corpus.
Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual architecture that operates on a saliency map and uses a novel attention mechanism to sequentially focus on salient regions and take additional glimpses within those regions. The architecture is motivated by human visual attention, and is used for multi-label image classification on a novel multiset task, demonstrating that it achieves high precision and recall while localizing objects with its attention. Unlike conventional multi-label image classification models, the model supports multiset prediction due to a reinforcement-learning based training process that allows for arbitrary label permutation and multiple instances per label.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Luc Van Gool
- Jitendra Malik
- Ming-Hsuan Yang
- Andrea Vedaldi
- Trevor Darrell
- Xiaogang Wang
- Andrew Zisserman
Citations in the corpus
- Object-Centric Learning with Slot Attention (2020, methodology): “[79] Sean Welleck, Jialin Mao, Kyunghyun Cho, and Zheng Zhang.”
- Deep Reinforcement Learning: An Overview (2017, methodology): “s with DQN.Mathe et al. (2016) proposed to use policy search for visual object detection.Kong et al.(2017) deployed collaborative multi-agent RL with inter-agent communication for joint object search.Welleck et al. (2017) proposed a hierarchical visual architecture with an attention mechanism for mu”
- Loss Functions for Multiset Prediction (2017, methodology): “Reinforcement Learning In [25], an approach based on reinforcement learning (RL) was proposed for multiset prediction.”
- Object Detection With Deep Learning: A Review (2018, methodology): “For scale-adaptive detectors, it is useful to combine knowledge graph [215], attentional mechanism [216], cascade network [180] and scale distribution estimation [171] to detect objects adaptively.”