BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

By Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, Nong Sang. 2018

2,729 citations recorded by the source corpus.

Semantic segmentation requires both rich spatial information and sizeable receptive field. However, modern approaches usually compromise spatial resolution to achieve real-time inference speed, which leads to poor performance. In this paper, we address this dilemma with a novel Bilateral Segmentation Network (BiSeNet). We first design a Spatial Path with a small stride to preserve the spatial information and generate high-resolution features. Meanwhile, a Context Path with a fast downsampling strategy is employed to obtain sufficient receptive field. On top of the two paths, we introduce a new Feature Fusion Module to combine features efficiently. The proposed architecture makes a right balance between the speed and segmentation performance on Cityscapes, CamVid, and COCO-Stuff datasets. Specifically, for a 2048 \(\times \) 1024 input, we achieve 68.4% Mean IOU on the Cityscapes test dataset with speed of 105 FPS on one NVIDIA Titan XP card, which is significantly faster than the existing methods with comparable performance.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “BiSeNet [35] was designed for real-time semantic segmentation.”
Rewriting a Deep Generative Model (2020, methodology): “For faces, we segment the image using a modiﬁed BiSeNet [79] as published by ZLL as faceparsing-Pytorch [87].”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, methodology): “BiSeNet (Yu et al., 2018) and HRNet (Wang et al., 2019) design multiple paths to learn both high-and low-resolution representations for better dense prediction.”
Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation (2022, background): “In the recent years, we have witnessed substantial progress in this area [1], [2], [3], [4], [5], [6], mostly thanks to the introduction of fully convolutional networks [1] and to the integration of several other architectural components, such as attention models [7], [8], [9] or multi-scale fusion ”
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers (2020, neutral): “…(i) Progressively upsampling the feature maps, SETR-Method Backbone mIoU PSPNet [60] ResNet-101 78.40 DenseASPP [50] DenseNet-161 80.60 BiSeNet [52] ResNet-101 78.90 PSANet [61] ResNet-101 80.10 DANet [18] ResNet-101 81.50 OCNet [55] ResNet-101 80.10 CCNet [25] ResNet-101 81.90 Axial-DeepLab-L…”
Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “BiSeNet [166] ResNet-101 78.”
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation (2020, methodology): “To obtain semantic maps, we use a BiSeNet model [36] pretrained on CelebAMask-HQ [19], and a unified parsing network [33] pretrained on Broden+ [4].”
SPGNet: Semantic Prediction Guidance for Scene Parsing (2019, neutral): “3B BiSeNet [70] ResNet-101 78.”
OCNet: Object Context Network for Scene Parsing (2018, neutral): “1 BiSeNet [21]‡ ECCV2018 ResNet-101 78.”
Image Segmentation Using Deep Learning: A Survey (2020, background): “Net) [128], Semantic Prediction Guidance for Scene Parsing (SPGNet) [129], gated shape CNNs [130], Adaptive Context”
SofGAN: A Portrait Image Generator with Dynamic Styling (2020, methodology): “12 segmentation maps for each frame with a pre-trained face parser [Yu et al. 2018].”
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis (2020, methodology): “Compared to other methods, our ESANet lies in between mobile (SwiftNet, BiSeNet) and non-mobile approaches for both mIoU and inference time.”
Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision (2020, background): “The leaderboard for one of the largest publicly available datasets, Modd2 [188], also contains a listing of semantic segmentation method capable to perform in marine environment [186], [187], [189]–[194].”
BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation (2020, methodology): “…al., 2016), ESPNet (Mehta et al., 2018), ESPNetV2 (Mehta et al., 2019), ERFNet (Romera et al., 2018), Fast-SCNN (Poudel et al., 2019), ICNet (Zhao et al., 2018a), DABNet (Li et al., 2019a), DFANet (Li et al., 2019b), GUN (Mazzini, 2018), SwiftNet (Or-sic et al., 2019), BiSeNetV1 (Yu et al., 2018a).”
FasterSeg: Searching for Faster Real-time Semantic Segmentation (2019, methodology): “Since no previous work has considered real-time segmentation on the BDD dataset, we get 15 times faster than the DRN-D-22 with slightly higher mIoU.”