FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation

By Huikai Wu, Junge Zhang, Kaiqi Huang, Kongming Liang, Yizhou Yu. 2019

340 citations recorded by the source corpus.

Modern approaches for semantic segmentation usually employ dilated convolutions in the backbone to extract high-resolution feature maps, which brings heavy computation complexity and memory footprint. To replace the time and memory consuming dilated convolutions, we propose a novel joint upsampling module named Joint Pyramid Upsampling (JPU) by formulating the task of extracting high-resolution feature maps into a joint upsampling problem. With the proposed JPU, our method reduces the computation complexity by more than three times without performance loss. Experiments show that JPU is superior to other upsampling modules, which can be plugged into many existing approaches to reduce computation complexity and improve performance. By replacing dilated convolutions with the proposed JPU module, our method achieves the state-of-the-art performance in Pascal Context dataset (mIoU of 53.13%) and ADE20K dataset (final score of 0.5584) while running 3 times faster.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Danfeng Hong
2. Yuhui Ma
3. Fei Shi
4. Dehui Xiang
5. Jing Yao
6. Shuanglang Feng
7. Meng Wang
8. Chenyu Li

In Computer Vision and Pattern Recognition

1. Danfeng Hong
2. Jing Yao
3. Chenxi Duan
4. Bing Zhang
5. Xiaoxiang Zhu
6. Yuhui Ma
7. Fei Shi
8. Dehui Xiang

In Model Architectures

1. Yuhui Ma
2. Fei Shi
3. Dehui Xiang
4. Jing Yao
5. Shuanglang Feng
6. Meng Wang
7. Shunyi Zheng
8. Danfeng Hong

Citations in the corpus

• UIU-Net: U-Net in U-Net for Infrared Small Object Detection (2022, neutral): “framework [35] to dilated/atrous framework [36].”
• Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks (2023, background): “The [23], [17], [16], [24] and [12] models fail to consider data shifts between different domains, while the rest effectively embed the DA strategy into networks.”
• Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images (2020, methodology): “FastFCN [36] further replaces the dilated convolutions with a joint pyramid upsampling (JPU) module to reduce computational complex-”
• CPFNet: Context Pyramid Fusion Network for Medical Image Segmentation (2020, methodology): “Fig.1 demonstrates the proposed CPFNet, which is an FCN based on encoder-decoder architecture and consists of four main parts: feature encoder, GPG module, SAPF module and feature decoder.”
• Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation (2022, neutral): “The following methods [16, 25, 38, 39, 44] are mainly based on FCN and enlarge the perceptual field, making it more suitable for semantic segmentation tasks that require intensive annotation.”