Adaptive Pyramid Context Network for Semantic Segmentation

By Junjun He, Zhongying Deng, Lei Zhou, Yali Wang, Yu Qiao. 2019

384 citations recorded by the source corpus.

Recent studies witnessed that context features can significantly improve the performance of deep semantic segmentation networks. Current context based segmentation methods differ with each other in how to construct context features and perform differently in practice. This paper firstly introduces three desirable properties of context features in segmentation task. Specially, we find that Global-guided Local Affinity (GLA) can play a vital role in constructing effective context features, while this property has been largely ignored in previous works. Based on this analysis, this paper proposes Adaptive Pyramid Context Network (APCNet) for semantic segmentation. APCNet adaptively constructs multi-scale contextual representations with multiple well-designed Adaptive Context Modules (ACMs). Specifically, each ACM leverages a global image representation as a guidance to estimate the local affinity coefficients for each sub-region, and then calculates a context vector with these affinities. We empirically evaluate our APCNet on three semantic segmentation and scene parsing datasets, including PASCAL VOC 2012, Pascal-Context, and ADE20K dataset. Experimental results show that APCNet achieves state-of-the-art performance on all three benchmarks, and obtains a new record 84.2% on PASCAL VOC 2012 test set without MS COCO pre-trained and any post-processing.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “W 48 achieves state-of-the-art results except that the result from [51] is higher than ours without using the OCR scheme [170].”
Object-Contextual Representations for Semantic Segmentation (2019, background): “For example, the ACFNet [22] exploits both the multi-scale context and relational context to achieve higher performance.”
Segmenter: Transformer for Semantic Segmentation (2021, neutral): “…48.5 DANet [23] ResNet-101 52.6 ANN [69] ResNet101 52.8 CPNet [58] ResNet-101 53.9 CFNet [64] ResNet-101 54.0 ACNet [24] ResNet-101 54.1 APCNet [26] ResNet101 54.7 DNL [57] HRNetV2-W48 55.3 DRANet [22] ResNet-101 55.4 OCR [60] HRNetV2-W48 5 shows a qualitative comparison of Segmenter and…”
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021, background): “29 ]; introducing boundary information [30–37]; designing various attention modules [38–46]; or using AutoML technologies [47–51].”
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing (2020, methodology): “Another line of research on improving the receptive field is the spatial pyramid pooling [65, 19].”
OCNet: Object Context Network for Scene Parsing (2018, neutral): “89 APCNet (He et al., 2019) ResNet-101 45.”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “Other models use multiscale analysis for segmentation, among them Dynamic Multiscale Filters Network (DMNet) [56], Context Contrasted Network and gated multiscale aggregation (CCN) [57], Adaptive Pyramid Context Network (APC-Net) [58], MultiScale Context Intertwining (MSCI) [59], and salient object ”
Semantic Flow for Fast and Accurate Scene Parsing (2020, methodology): “re contextual information. PSPNet [67] proposes to leverage pyramid pooling module (PPM) to model multi-scale contexts, whilst DeepLab series [5{7, 55] uses astrous spatial pyramid pooling (ASPP). In [16, 18, 19, 21, 29, 59, 72], non-local operator [52] and self-attention mechanism [51] are adopted ”
Dynamic Multi-Scale Filters for Semantic Segmentation (2019, methodology): “[20] utilizes pyramid context to construct multi-scale representations, which are adapted to input images.”
A Survey on Vision Transformer (2020, background): “PSANet [151], OCNet [139], DANet [38] and CFNet [147] are the first works that introduce the self-attention module to semantic segmentation task, which consider and augment the relation and similarity [146, 74, 46, 89, 130] between the contextual pixels.”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks (2021, background): “Method SA [16] DA [4] A(2) [94] APC [76] DM [95] ACF [96] Ham [8] EA (ours)”
Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, positive): “In [10,35,12,36,11,13], non-local operator [37] and self-attention mechanism [38] are adopted to harvest pixel-wise context from the whole image.”