Dynamic Computational Time for Visual Attention

By Zhichao Li, Yi Yang, Xiao Liu, Feng Zhou, Shilei Wen, Wei Xu. 2017

117 citations recorded by the source corpus.

We propose a dynamic computational time model to accelerate the average processing time for recurrent visual attention (RAM). Rather than attention with a fixed number of steps for each input image, the model learns to decide when to stop on the fly. To achieve this, we add an additional continue/stop action per time step to RAM and use reinforcement learning to learn both the optimal attention policy and stopping policy. The modification is simple but could dramatically save the average computational time while keeping the same recognition performance as RAM. Experimental results on CUB-200-2011 and Stanford Cars dataset demonstrate the dynamic computational model can work effectively for fine-grained image recognition. The source code of this paper can be obtained from https://github.com/baidu-research/DT-RAM

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, methodology): “The recurrent attention mechanism together with the early exiting paradigm enables both spatially and temporally adaptive inference [39], [160].”
Improved Techniques for Training Adaptive Deep Networks (2019, background): “Recently, a new emerging direction which employs adaptive learning for efﬁcient inference has drawn increasing research attention, with representative works proposed in [14, 2, 25, 35, 19, 6, 24, 38, 36, 29, 37, 18].”
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition (2018, methodology): “We also note that although a better ResNet-50 baseline on CUB is reported in [35] (84.5%), it is implemented in Torch [53] and tuned with more advanced data augmentation ( e.g. , color jittering, scaling).”
Differentiable Patch Selection for Image Recognition (2021, methodology): “These methods often rely on Reinforcement Learning to train non-differentiable attention mechanisms [33, 4, 28, 12], which typically makes them difficult to train.”
Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks (2018, methodology): “Compared to DT-RAM [33], one of the top performing models in CUB-200, our approach outperforms the comparable 224 × 224 version of RN-50 DT-RAM by 1.7%, using a simpler model.”
Recurrence is required to capture the representational dynamics of the human visual system (2019, unknown): “35 Table 1 Network architectural parameters Layer Feature maps Image size BK9 – kernel size (effective) BK11 – kernel size (effective) BLT – kernel size (effective) 1 64 96 × 96 9 (9) 11 (11) 5 (5) 2 64 48 × 48 9 (9) 11 (11) 5 (5) 3 96 24 × 24 9 (9) 11 (11) 5 (5) 4 96 12 × 12 9 (9) 11 (11) 5 (5) 5 1”
Learning to Navigate for Fine-grained Classification (2018, methodology): “6% DT-RAM [26] 86 .”