Rethinking BiSeNet For Real-time Semantic Segmentation

By Mingyuan Fan, Shenqi Lai, Junshi Huang, Xiaoming Wei, Zhenhua Chai, Junfeng Luo, Xiaolin Wei. Computer Vision and Pattern Recognition, 2021

709 citations recorded by the source corpus.

BiSeNet [28], [27] has been proved to be a popular two-stream network for real-time segmentation. However, its principle of adding an extra path to encode spatial information is time-consuming, and the backbones borrowed from pretrained tasks, e.g., image classification, may be inefficient for image segmentation due to the deficiency of task-specific design. To handle these problems, we propose a novel and efficient structure named Short-Term Dense Concatenate network (STDC network) by removing structure redundancy. Specifically, we gradually reduce the dimension of feature maps and use the aggregation of them for image representation, which forms the basic module of STDC network. In the decoder, we propose a Detail Aggregation module by integrating the learning of spatial information into low-level layers in single-stream manner. Finally, the low-level features and deep features are fused to predict the final segmentation results. Extensive experiments on Cityscapes and CamVid dataset demonstrate the effectiveness of our method by achieving promising trade-off between segmentation accuracy and inference speed. On Cityscapes, we achieve 71.9% mIoU on the test set with a speed of 250.4 FPS on NVIDIA GTX 1080Ti, which is 45.2% faster than the latest methods, and achieve 76.8% mIoU with 97.0 FPS while inferring on higher resolution images. Code is available at https://github.com/MichaelFan01/STDC-Seg.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e51b5153df01ed6b634cfc6180661cba5c84e62b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, neutral): “STDC [19] proposes a novel and eﬃcient structure named Short-Term Dense Concatenate network by removing structure redundancy.”
Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes (2023, methodology): “Nevertheless, Fan et al. [24] design more powerful pre-trained backbones to highlight their signiﬁcance and discard the detail branch.”
PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers (2022, background): “Our training protocols are almost the same as previous works [15, 20, 52].”
SegDiff: Image Segmentation with Diffusion Probabilistic Models (2021, methodology): “We compared our method for the Cityscapes dataset with PSPDeepLab [5], Polygon-RNN++ [1], Curve-GCN [29] Deep active contours [14], Segformer-B5 [50] and Stdc1 [11].”
ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation (2022, background): “Semantic segmentation is one of the fundamental tasks in the computer vision field, which aims to predict the category of each pixel of an image [7, 15, 31, 41].”