Dual Attention Network for Scene Segmentation

By Jun Fu, J. Liu, Haijie Tian, Zhiwei Fang, Hanqing Lu. 2018

6,139 citations recorded by the source corpus.

In this paper, we address the scene segmentation task by capturing rich contextual dependencies based on the self-attention mechanism. Unlike previous works that capture contexts by multi-scale features fusion, we propose a Dual Attention Networks (DANet) to adaptively integrate local features with their global dependencies. Specifically, we append two types of attention modules on top of traditional dilated FCN, which model the semantic interdependencies in spatial and channel dimensions respectively. The position attention module selectively aggregates the features at each position by a weighted sum of the features at all positions. Similar features would be related to each other regardless of their distances. Meanwhile, the channel attention module selectively emphasizes interdependent channel maps by integrating associated features among all channel maps. We sum the outputs of the two attention modules to further improve feature representation which contributes to more precise segmentation results. We achieve new state-of-the-art segmentation performance on three challenging scene segmentation datasets, i.e., Cityscapes, PASCAL Context and COCO Stuff dataset. In particular, a Mean IoU score of 81.5% on Cityscapes test set is achieved without using coarse data.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Rethinking Attention with Performers (2020, background): “FAVOR+ can be also applied beyond the Transformer scope as a more scalable replacement for regular attention, which itself has a wide variety of uses in computer vision (Fu et al., 2019), reinforcement learning (Zambaldi et al.”
Exploring Self-Attention for Image Recognition (2020, methodology): “Other approaches [4, 36, 8] adopted both spatial and channel attention.”
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, background): “On semantic segmentation, self-attention is developed as a context aggregation module that captures multi-scale context [42,27,102,99].”
CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “OCNet [38] and DANet [15] utilized Non-local module [31] to harvest the contextual information.”
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021, background): “The selfattention layers can complement backbones [61, 7, 3, 65, 21, 68, 51] or head networks [29, 24] by providing the capability to encode distant dependencies or heterogeneous interactions.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, background): “Both classes of methods have boosted various vision tasks such as image classification [18, 5, 72, 37, 57, 89, 28], object detection [91, 80, 72, 36, 10, 108], semantic segmentation [15, 106, 39, 29, 109, 107], video recognition [91, 18], image generation [73, 35], and panoptic segmentation [89].”
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, background): “In the recent years, we have witnessed substantial progress in this area [1], [2], [3], [4], [5], [6], mostly thanks to the introduction of fully convolutional networks [1] and to the integration of several other architectural components, such as attention models [7], [8], [9] or multi-scale fusion ”
Representation Compensation Networks for Continual Semantic Segmentation (2022, methodology): “Some methods [14,22,32,37,38,50,55] is inspired by Non-local [85], utilizing attention mechanisms to establish connections between image contexts.”
PLOP: Learning without Forgetting for Continual Semantic Segmentation (2020, background): “Examples of recent works exploiting contextual information include attention mechanisms [69, 76, 22, 30, 68, 61, 73], and fixed-scale aggregation [75, 12, 11, 72].”
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, background): “DANet [16] embeds both spatial attention and channel attention.”
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet (2021, methodology): “Compared with most of the work exploring the global attention on images [1, 42, 13, 39], some works [18, 31] also explore the self-attention in a local patch to reduce the memory and computation cost.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “ResNet-101 - 45:7 ReﬁneNet [70] ResNet-152 - 47:3 UNet++ [151] ResNet-101 47:7 - PSPNet [144] Dilated-ResNet-101 47:8 - Ding et al. [25] ResNet-101 51:6 - EncNet [137] Dilated-ResNet-101 52:6 - DANet [30] Dilated-ResNet-101 52:6 - SVCNet [26] ResNet-101 53:2 - CFNet [138] Dilated-ResNet-101 54:0 - A”
Object-Contextual Representations for Semantic Segmentation (2019, methodology): “On the other hand, our OCR approach is also different from the previous relational context schemes [55, 15, 60, 61, 63].”
Segmenter: Transformer for Semantic Segmentation (2021, methodology): “Recent work [22, 23, 57, 58, 61, 66] replace coarse pooling by attention mechanisms on top of the encoder feature maps to better capture long-range dependencies.”
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021, background): “29 ]; introducing boundary information [30–37]; designing various attention modules [38–46]; or using AutoML technologies [47–51].”
ResNeSt: Split-Attention Networks (2020, background): “P ri o r W o rk DANet [16] ResNet101 77.”