Dynamic Multi-Scale Filters for Semantic Segmentation

By Junjun He, Zhongying Deng, Yu Qiao. 2019

288 citations recorded by the source corpus.

Multi-scale representation provides an effective way to address scale variation of objects and stuff in semantic segmentation. Previous works construct multi-scale representation by utilizing different filter sizes, expanding filter sizes with dilated filters or pooling grids, and the parameters of these filters are fixed after training. These methods often suffer from heavy computational cost or have more parameters, and are not adaptive to the input image during inference. To address these problems, this paper proposes a Dynamic Multi-scale Network (DMNet) to adaptively capture multi-scale contents for predicting pixel-level semantic labels. DMNet is composed of multiple Dynamic Convolutional Modules (DCMs) arranged in parallel, each of which exploits context-aware filters to estimate semantic representation for a specific scale. The outputs of multiple DCMs are further integrated for final segmentation. We conduct extensive experiments to evaluate our DMNet on three challenging semantic segmentation and scene parsing datasets, PASCAL VOC 2012, Pascal-Context, and ADE20K. DMNet achieves a new record 84.4% mIoU on PASCAL VOC 2012 test set without MS COCO pre-trained and post-processing, and also obtains state-of-the-art performance on Pascal-Context and ADE20K.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

K-Net: Towards Unified Image Segmentation (2021, methodology): “Some semantic segmentation methods apply dynamic kernels to improve the model representation with enlarged receptive fields [56] or multi-scales contexts [20].”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “Other models use multiscale analysis for segmentation, among them Dynamic Multiscale Filters Network (DMNet) [56], Context Contrasted Network and gated multiscale aggregation (CCN) [57], Adaptive Pyramid Context Network (APC-Net) [58], MultiScale Context Intertwining (MSCI) [59], and salient object ”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, background): “In [10,35,12,36,11,13], non-local operator [37] and self-attention mechanism [38] are adopted to harvest pixel-wise context from the whole image.”
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation (2020, methodology): “Besides exploiting various contextual information from the visual cues [25,11,14,12,5,43], depth data have recently been utilized as supplementary information to RGB data to achieve improved segmentation accuracy [26,33,44,4,15,23,6,19].”
Disentangled Non-Local Neural Networks (2020, methodology): “…[41] ResNet-101 44.65 55.67 52.6 DANet [13] ResNet-101 45.22 - 52.6 CFNet [42] ResNet-101 44.89 - 54.0 ANN [47] ResNet-101 45.24 - 52.8 DMNet [17] ResNet-101 latency (single-scale inference using a single GPU) on semantic segmentation, using a ResNet-101 backbone and input resolution of 769…”
Semantic Flow for Fast and Accurate Scene Parsing (2020, background): “In [13,15,16,18,25,53,66], non-local operator [46] and self-attention mechanism [45] are adopt to harvest pixel-wise context from whole image.”