Rethinking Atrous Convolution for Semantic Image Segmentation

By Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. 2017

10,062 citations recorded by the source corpus.

In this work, we revisit atrous convolution, a powerful tool to explicitly adjust filter's field-of-view as well as control the resolution of feature responses computed by Deep Convolutional Neural Networks, in the application of semantic image segmentation. To handle the problem of segmenting objects at multiple scales, we design modules which employ atrous convolution in cascade or in parallel to capture multi-scale context by adopting multiple atrous rates. Furthermore, we propose to augment our previously proposed Atrous Spatial Pyramid Pooling module, which probes convolutional features at multiple scales, with image-level features encoding global context and further boost performance. We also elaborate on implementation details and share our experience on training our system. The proposed `DeepLabv3' system significantly improves over our previous DeepLab versions without DenseCRF post-processing and attains comparable performance with other state-of-art models on the PASCAL VOC 2012 semantic image segmentation benchmark.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “Recently, Panoptic-DeepLab [18], a simple, fast, and strong approach for bottom-up panoptic segmentation, employs a class-agnostic instance segmentation branch involving a simple instance center regression [41,77,61], coupled with DeepLab semantic segmentation outputs [12,14,15].”
Weight Standardization (2019, methodology): “We select DeepLabv3 [5] as the base model for its competitive performance and use ResNet-101 as backbone.”
CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “[6] used different dilation convolutions to harvest pixel-wise contextual information at the different range, donated as “+ASPP”; 4) Wang et al .”
Bag of Tricks for Image Classification with Convolutional Neural Networks (2018, methodology): “The idea of label smoothing was ﬁrst proposed to train Inception-v2 [26].”
UPSNet: A Unified Panoptic Segmentation Network (2019, background): “Recent semantic segmentation methods that exploit deep convolutional feature extraction mainly approach this problem from either multi-scale feature aggregation [40, 26, 5, 14], or end-to-end structured prediction [2, 41, 1, 25, 4] perspectives.”
Panoptic Feature Pyramid Networks (2019, background): “backbone mIoU FLOPs memory DeeplabV3 [11] ResNet-101-D8 77.”
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs (2021, positive): “We choose again the DeepLabv3-ResNet-50 [4] semantic segmentation network, trained on the Cityscapes [7] dataset.”
Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement (2023, methodology): “To evaluate the transferability of models pre-trained using ImageNet + dataset, we evaluate on following tasks: (1) semantic segmentation with DeepLabv3 [11] on the ADE20K dataset [74], (2) object detection with Mask-RCNN [20] on the MS-COCO dataset [34], and (3) fine-grained classification on the C”
Generalized Decoding for Pixel, Image, and Language (2022, background): “Semantic segmentation cares about the per-pixel semantic within an image [6,11,48], whereas instance segmentation groups pixels of the same semantic meaning into objects.”
Language-driven Semantic Segmentation (2022, neutral): “We compare the results to highly competitive standard semantic segmentation models, including OCNet (Yuan et al., 2020), ACNet (Fu et al., 2019), DeeplabV3 (Chen et al., 2017; Zhang et al., 2020a), and DPT (Ranftl et al., 2021).”
Masked-attention Mask Transformer for Universal Image Segmentation (2021, methodology): “Follow-up methods find context to play an important role for precise per-pixel classification and focus on designing customized context modules [7,8,63] or self-attention variants [21,26,45,55,61,64].”
Per-Pixel Classification is Not All You Need for Semantic Segmentation (2021, methodology): “We use a ResNet-50 backbone and report single scale mIoU and PQSt for ADE20K, COCO-Stuff and ADE20K-Full, whereas for higher-resolution Cityscapes we use a deeper ResNet-101 backbone following [8, 9].”
K-Net: Towards Unified Image Segmentation (2021, methodology): “As K-Net does not rely on specific architectures of model representation, K-Net can perform semantic segmentation by simply appending its kernel update head to any existing semantic segmentation methods [8, 40, 59, 69] that rely on semantic kernels.”
ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes (2021, background): “Recent semantic segmentation models [12,37,62,63] have achieved state-of-the-art performance for recognizing general object/stuff classes from nat-ural scene images.”
Iteratively Trained Interactive Segmentation (2018, background): “The backbone DeepLabV3 [7] structure acts as an encoder module to which an additional decoder module is added to recover fine structures.”
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, methodology): “Spatial pyramid pooling: Models, such as PSPNet [81] or DeepLab [9, 10], perform spatial pyramid pooling [23, 40] at several grid scales (including image-level pooling [47]) or apply several parallel atrous convolution with different rates (called Atrous Spatial Pyramid Pooling, or ASPP).”
MobileNetV2: Inverted Residuals and Linear Bottlenecks (2018, methodology): “To build a mobile model, we experimented with three design variations: (1) different feature extractors, (2) simplifying the DeepLabv3 heads for faster computation, and (3) different inference strategies for boosting the performance.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Contrary to box-based approaches, box-free methods typically start with semantic segments [12, 14, 16].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “CNNs have shown incredible successes in computer vision tasks, such as image classification [1], [39], object detection [4], [40], text recognition [41], [42], and semantic segmentation [43], [44].”
Are Labels Necessary for Neural Architecture Search? (2020, methodology): “Different from the network used in IN1K classification, we (1) replace the last stride 2 layer with stride 1, to increase spatial resolution; and (2) remove the global average pooling and replace the fully connected classifier with the Atrous Spatial Pyramid Pooling (ASPP) module [1] followed by bil”