Scale-Adaptive Convolutions for Scene Parsing

By Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, Shuicheng Yan. 2017

196 citations recorded by the source corpus.

Many existing scene parsing methods adopt Convolutional Neural Networks with fixed-size receptive fields, which frequently result in inconsistent predictions of large objects and invisibility of small objects. To tackle this issue, we propose a scale-adaptive convolution to acquire flexiblesize receptive fields during scene parsing. Through adding a new scale regression layer, we can dynamically infer the position-adaptive scale coefficients which are adopted to resize the convolutional patches. Consequently, the receptive fields can be adjusted automatically according to the various sizes of the objects in scene images. Thus, the problems of invisible small objects and inconsistent large-object predictions can be alleviated. Furthermore, our proposed scale-adaptive convolutions are not only differentiable to learn the convolutional parameters and scale coefficients in an end-to-end way, but also of high parallelizability for the convenience of GPU implementation. Additionally, since the new scale regression layers are learned implicitly, any extra training supervision of object sizes is unnecessary. Extensive experiments on Cityscapes and ADE20K datasets well demonstrate the effectiveness of the proposed scaleadaptive convolutions.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Chunhua Shen
4. Jingdong Wang
5. Xiaogang Wang
6. Ming-Hsuan Yang
7. Yu Qiao
8. Yi Yang

Citations in the corpus

• CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “SAC [40] and Deformable Convolutional Networks [11] improved the standard convolutional operator to handle the deformation and various scale of objects.”
• Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “et [90] ResNet-101 73:6 47:2 87:9 70:6 PEARL [65] D-ResNet-101 75:4 51:6 89:2 75:1 DSSPN [88] D-ResNet-101 76:6 56:2 89:6 77:8 LKM [111] ResNet-152 76:9 - - - DUC-HDC [144] - 77 :6 53 6 90 1 75 2 SAC [176] D-ResNet-101 78:1 - - - DepthSeg [73] D-ResNet-101 78:2 - - - ResNet38 [151] WResNet-38 78:4 5”
• BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation (2018, background): “[38] improves the neural network by a scale adaptive convolution layer to obtain an adaptive field context information.”
• Image Segmentation Using Deep Learning: A Survey (2020, background): “In another relevant effort, Marcos et al. [113] proposed Deep Structured Active Contours (DSAC), which combines ACMs and pre-trained FCNs in a structured prediction framework for building instance segmentation (albeit with manual initialization) in aerial images.”
• Dense Relation Network: Learning Consistent and Context-Aware Representation for Semantic Image Segmentation (2018, unknown): “2 SAC multiple [31] 78.”
• Scale equivariance in CNNs with vector fields (2018, background): “For instance, in histology and cytology, the sizes of cells and organelles can provide hints about their functionality, while in scene understanding, apparent object sizes implicitly encode distance from the camera and can be valuable information for scene segmentation (Zhang et al., 2017).”
• CGNet: A Light-Weight Context Guided Network for Semantic Segmentation (2018, background): “SAC [37] proposes a scale-adaptive convolution to acquire flexiblesize receptive fields.”
• Learning Semantic Neural Tree for Human Parsing (2019, unknown): “81 SS-NAN [36] 87.”
• Dilated SpineNet for Semantic Segmentation (2021, methodology): “[29, 48, 3, 4, 5, 6, 44, 52] advocate to use dilated convolution at certain stages of existing architectures to expand receptive filed of convolution kernels without downsampling.”
• Adaptive Pyramid Context Network for Semantic Segmentation (2019, unknown): “…32.31 CascadeNet [42] 34.90 RefineNet [17] ResNet152 40.7 PSPNet [40] ResNet101 43.29 PSPNet [40] ResNet269 44.94 EncNet [38] ResNet101 44.65 SAC [39] ResNet101 44.30 PSANet [13] ResNet101 43.77 UperNet [33] ResNet101 42.66 DSSPN [15] ResNet101 43.68 OCNet [37] ResNet101 45.08 Ours ResNet101…”