Pixel-Wise Attentional Gating for Scene Parsing
By Shu Kong, Charless C. Fowlkes. 2018
43 citations recorded by the source corpus.
To achieve dynamic inference in pixel labeling tasks, we propose Pixel-wise Attentional Gating (PAG), which learns to selectively process a subset of spatial locations at each layer of a deep convolutional network. PAG is a generic, architecture-independent, problem-agnostic mechanism that can be readily "plugged in" to an existing model with fine-tuning. We utilize PAG in two ways: 1) learning spatially varying pooling fields that improve model performance without the extra computation cost associated with multi-scale pooling, and 2) learning a dynamic computation policy for each pixel to decrease total computation (FLOPs) while maintaining accuracy. We extensively evaluate PAG on a variety of per-pixel labeling tasks, including semantic segmentation, boundary detection, monocular depth and surface normal estimation. We demonstrate that PAG allows competitive or state-of-the-art performance on these tasks. Our experiments show that PAG learns dynamic spatial allocation of computation over the input image which provides better performance trade-offs compared to related approaches (e.g., truncating deep models or dynamically skipping whole layers). Generally, we observe PAG can reduce computation by 10% without noticeable loss in accuracy and performance degrades gracefully when imposing stronger computational constraints.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Shiji Song
- Yulin Wang
- Gao Huang
- Le Yang
- Yizeng Han
- Honghui Wang
- Esther Luna Colombini
- Alana de Santana Correia
In Computer Vision and Pattern Recognition
- Shiji Song
- Yulin Wang
- Gao Huang
- Yizeng Han
- Le Yang
- Honghui Wang
- Esther Luna Colombini
- Alana de Santana Correia
Citations in the corpus
- Dynamic Neural Networks: A Survey (2021, background): “elements on the output [144], [145]; and 3) estimating the saliency of pixels [146], [147], [148].”
- Attention, please! A survey of neural attention models in deep learning (2021, background): “(2019g), Kong and Fowlkes (2019), Li and Loy (2018) and Zhao et al.”