Fast-SCNN: Fast Semantic Segmentation Network

By Rudra P. K. Poudel, Stephan Liwicki, Roberto Cipolla. British Machine Vision Conference, 2019

692 citations recorded by the source corpus.

The encoder-decoder framework is state-of-the-art for offline semantic image segmentation. Since the rise in autonomous systems, real-time computation is increasingly desirable. In this paper, we introduce fast segmentation convolutional neural network (Fast-SCNN), an above real-time semantic segmentation model on high resolution image data (1024x2048px) suited to efficient computation on embedded devices with low memory. Building on existing two-branch methods for fast segmentation, we introduce our `learning to downsample' module which computes low-level features for multiple resolution branches simultaneously. Our network combines spatial detail at high resolution with deep features extracted at lower resolution, yielding an accuracy of 68.0% mean intersection over union at 123.5 frames per second on Cityscapes. We also show that large scale pre-training is unnecessary. We thoroughly validate our metric in experiments with ImageNet pre-training and the coarse labeled data of Cityscapes. Finally, we show even faster computation with competitive results on subsampled inputs, without any network modifications.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b2324651155468c9b6bef8a2e006272126d17608

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Song Han
3. Chuang Gan
4. Barret Zoph
5. Mingxing Tan
6. Hanxiao Liu
7. Ekin Dogus Cubuk
8. Han Cai

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Quoc V. Le
3. Dacheng Tao
4. Xiaogang Wang
5. Ming-Hsuan Yang
6. Alan L. Yuille
7. Trevor Darrell
8. Jitendra Malik

Citations in the corpus

• SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds (2020, positive): “n is the lack of contextual information throughout the network. The global context information gathered by larger receptive fields plays a crucial role in learning complex correlations between classes [5]. To aggregate the context information in different regions, we fuse a larger receptive field with”
• EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction (2023, background): “In addition, there are also some works targeting improving the efficiency of high-resolution dense prediction models [57, 40, 29, 52].”
• Continual Learning of Semantic Segmentation using Complementary 2D-3D Data Representations (2021, methodology): “We use Fast-SCNN [23] as a semantic segmentation network.”
• PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers (2022, background): “Recently, many novel and promising models based on Two-Branch Network (TBN) architecture have been proposed in the literature and achieve SOTA trade-off between speed and accuracy [15, 20, 38, 39, 52].”
• Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, positive): “However, due to the small amount of data and the similar spectra of the two kinds of ice, the accuracy of ContextNet [30] and FastSCNN [18] are very low.”
• Knowledge Distillation for Semantic Segmentation Using Channel and Spatial Correlations and Adaptive Cross Entropy (2020, methodology): “Fast-SCNN [41] employs a pseudo two-branch architecture using the skip connection.”
• BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation (2020, neutral): “The real-time semantic segmentation algorithms are listed in the second group, including ENet (Paszke et al., 2016), SQ (Treml et al., 2016), ESPNet (Mehta et al., 2018), ESPNetV2 (Mehta et al., 2019), ERFNet (Romera et al., 2018), Fast-SCNN (Poudel et al., 2019), ICNet (Zhao et al., 2018a), DABNet ”
• UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery (2021, positive): “In comparison with the fastest and most shallow model Fast-SCNN, the proposed UNetFormer outperforms it by a large margin of 21.0% in mIoU.”
• FasterSeg: Searching for Faster Real-time Semantic Segmentation (2019, neutral): “…multi-scale, etc. (Zhao et al., 2018) 67.7 69.5 37.7 1024 × 2048 BiSeNet (Yu et al., 2018a) 69.0 68.4 105.8 768 × 1536 CAS (Zhang et al., 2019) 71.6 70.5 108.0 768 × 1536 Fast-SCNN (Poudel et al., 2019) CamVid: We directly transfer the searched architecture on Cityscapes to train on CamVid.”
• Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes (2023, methodology): “The two pathways in BiSeNetV1&V2 are initially separate while the two branches in Fast-SCNN [19] share the learning to downsample module.”
• CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers (2022, methodology): “We compare our models against RGB-only methods covering CNN-based [100], [117], [118], [120] and transformerbased [24], [46], [119] methods.”
• Rethinking Pre-training and Self-training (2020, background): “Poudel et al [43] on the other hand show that ImageNet pre-training is not necessary for semantic segmentation with CityScapes if aggressive data augmentation is applied.”