Asymmetric Non-Local Neural Networks for Semantic Segmentation

By Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, Xiang Bai. 2019

705 citations recorded by the source corpus.

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: https://github.com/MendelXu/ANN.git.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, background): “On semantic segmentation, self-attention is developed as a context aggregation module that captures multi-scale context [42,27,102,99].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, background): “Both classes of methods have boosted various vision tasks such as image classification [18, 5, 72, 37, 57, 89, 28], object detection [91, 80, 72, 36, 10, 108], semantic segmentation [15, 106, 39, 29, 109, 107], video recognition [91, 18], image generation [73, 35], and panoptic segmentation [89].”
Object-Contextual Representations for Semantic Segmentation (2019, background): “22 - - ANNet [83] ResNet-101 8× M,R 81.”
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing (2020, neutral): “1 APNB [69] ICCV’19 ResNet-101 52.”
OCNet: Object Context Network for Scene Parsing (2018, background): “Some recent efforts (Huang et al., 2019; Yue et al., 2018; Zhu et al., 2019) propose different mechanisms to decrease the computation complexity and memory consumption of selfattention scheme.”
Attention mechanisms in computer vision: A survey (2021, methodology): “[97, 46, 139, 70, 128, 36, 11, 156] , [137, 143, 63, 161, 159, 6, 177, 80] , [67, 180, 10, 23, 24, 163, 155, 151] , [44, 49, 106, 64, 168, 18, 11, 36] , [48, 16, 169, 153, 174, 127, 53, 92] , [89, 21, 5, 134, 65, 171, 2, 109] , [45, 82, 157, 91, 100, 129, 146, 131] , [138, 85, 175, 76, 166, 133, 167”
Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, positive): “In [10,35,12,36,11,13], non-local operator [37] and self-attention mechanism [38] are adopted to harvest pixel-wise context from the whole image.”
Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation (2022, neutral): “The following methods [16, 25, 38, 39, 44] are mainly based on FCN and enlarge the perceptual ﬁeld, making it more suitable for semantic segmentation tasks that require intensive annotation.”
LISA: Reasoning Segmentation via Large Language Model (2023, background): “Numerous studies (Shelhamer et al., 2017; Noh et al., 2015; Badrinarayanan et al., 2017; Ronneberger et al., 2015; Chen et al., 2018; Yu & Koltun, 2016; Liu et al., 2015; Zhao et al., 2017; 2018a; Yang et al., 2018; Fu et al., 2019; Huang et al., 2019; Zhao et al., 2018b; Zhu et al., 2019; Cheng et ”
OCNet: Object Context for Semantic Segmentation (2021, background): “Some recent efforts (Huang et al. 2019; Yue et al. 2018; Zhu et al. 2019) propose different mechanisms to decrease the computation complexity and memory consumption of self-attention scheme.”
Feature Pyramid Transformer (2020, neutral): “a wide range of visual tasks [11,32,19,33].”
UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation (2021, background): “Previous works attempt to reduce the complexity of self-attention [10,28], but are still far from perfection.”
Learning Statistical Texture for Semantic Segmentation (2021, methodology): “After that, a lot of methods [37, 3, 40, 21, 28, 22, 39] are proposed to improve the basic FCN.”
GFF: Gated Fully Fusion for Semantic Segmentation (2019, background): “ation is collected from all positions according to the similarities deﬁned in a projected feature space. Similarly, DANet (Fu et al. 2018), CCNet (Huang et al. 2018), EMAnet (Li et al. 2019) and ANN (Zhu et al. 2019) use non-local style operator (Wang et al. 2018) to aggregate information from the w”
Disentangled Non-Local Neural Networks (2020, neutral): “4 ANN [47] ResNet-101 81.”