Scene Segmentation With Dual Relation-Aware Attention Network

By Jun Fu, Jie Jiang, Yong Li, Yongjun Bao, Hanqing Lu. 2020

248 citations recorded by the source corpus.

In this article, we propose a Dual Relation-aware Attention Network (DRANet) to handle the task of scene segmentation. How to efficiently exploit context is essential for pixel-level recognition. To address the issue, we adaptively capture contextual information based on the relation-aware attention mechanism. Especially, we append two types of attention modules on the top of the dilated fully convolutional network (FCN), which model the contextual dependencies in spatial and channel dimensions, respectively. In the attention modules, we adopt a self-attention mechanism to model semantic associations between any two pixels or channels. Each pixel or channel can adaptively aggregate context from all pixels or channels according to their correlations. To reduce the high cost of computation and memory caused by the abovementioned pairwise association computation, we further design two types of compact attention modules. In the compact attention modules, each pixel or channel is built into association only with a few numbers of gathering centers and obtains corresponding context aggregation over these gathering centers. Meanwhile, we add a cross-level gating decoder to selectively enhance spatial details that boost the performance of the network. We conduct extensive experiments to validate the effectiveness of our network and achieve new state-of-the-art segmentation performance on four challenging scene segmentation data sets, i.e., Cityscapes, ADE20K, PASCAL Context, and COCO Stuff data sets. In particular, a Mean IoU score of 82.9% on the Cityscapes test set is achieved without using extra coarse annotated data.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training (2021, methodology): “In order to better aggregate semantic context or feature dependencies, other recent works [41, 10] employ self-attention modules on semantic segmentation networks.”
Segmenter: Transformer for Semantic Segmentation (2021, methodology): “Despite much effort and large progress over recent years [10, 22, 31, 37, 48, 65, 66], image segmentation remains a challenging problem due to rich intra-class variation, context variation and ambiguities originating from occlusions and low image resolution.”
SegDiff: Image Segmentation with Diffusion Probabilistic Models (2021, methodology): “Unlike other recent improvements in the ﬁeld of image segmentation [13, 22, 44], we train our method end-to-end, without relying on a pre-trained back-bone network.”