ICNet for Real-Time Semantic Segmentation on High-Resolution Images

By Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, Jiaya Jia. 2017

1,612 citations recorded by the source corpus.

We focus on the challenging task of real-time semantic segmentation in this paper. It finds many practical applications and yet is with fundamental difficulty of reducing a large portion of computation for pixel-wise label inference. We propose an image cascade network (ICNet) that incorporates multi-resolution branches under proper label guidance to address this challenge. We provide in-depth analysis of our framework and introduce the cascade feature fusion unit to quickly achieve high-quality segmentation. Our system yields real-time inference on a single GPU card with decent quality results evaluated on challenging datasets like Cityscapes, CamVid and COCO-Stuff.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

UPSNet: A Unified Panoptic Segmentation Network (2019, background): “Pyra-mid scene parsing network (PSPNet) [39] that uses dilated convolutions in its backbone, and its faster variant [38] for real-time applications are widely utilized in practical applications.”
AlterSGD: Finding Flat Minima for Continual Learning by Alternative Training (2021, background): “In addition, to improve the segmentation network efﬁciency for applications, light-weight design [33, 44] and network compression [13] can also incorporate and boost the advances for this task.”
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021, methodology): “On Citysapces, our lightweight model, SegFormer-B0, without accelerated implementations such as TensorRT, yields 71.9% mIoU at 48 FPS, which, compared to ICNet [11], represents a relative improvement of 60% and 4.2% in latency and performance, respectively.”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “Real-time Instance Segmentation While real-time object detection [30, 34, 35, 36], and semantic segmentation [2, 41, 33, 11, 47] methods exist, few works have focused on real-time instance segmentation.”
BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation (2018, methodology): “Recently, the algorithms [1,17,25,39] of real-time semantic segmentation have shown that there are mainly three approaches to accelerate the model.”
Using labeled data to evaluate change detectors in a multivariate streaming environment (2009, neutral): “Some researchers clip or resize the input image to reduce the computational complexity, such as SNet [13] and ICNet [14] .”
The ApolloScape Open Dataset for Autonomous Driving and Its Application (2018, methodology): “apply small models [60] or model compression [61] for acceleration, with the cost of reduced accuracy.”
Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery (2018, methodology): “A similar deﬁnition of ICN was proposed for real-time seman-Towards tic segmentation in [33], but without taking into account diﬀerent scales in the feature domain and using a cascaded label for each level to compensate for the sub-sampling.”
BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation (2020, methodology): “Facing this demand, based on both backbone networks, existing methods (Badrinarayanan et al., 2017; Paszke et al., 2016; Zhao et al., 2018a; Romera et al., 2018; Mazzini, 2018) mainly employ two appraches to accelerate the model: (i) Input Restricting .”
A survey of deep learning techniques for autonomous driving (2019, background): “, 2017), ICNet (Zhao et al., 2018a), ENet (Paszke et al.”
AI Benchmark: Running Deep Neural Networks on Android Smartphones (2018, methodology): “This is done with an ICNet CNN [67], designed for fast and accurate segmentation on low-performance devices.”
Survey on semantic segmentation using deep learning techniques (2019, background): “Zhao et al. [87] proposed image cascade network (ICNet) which utilizes semantic information in low resolution along with details from high-resolution images e ﬃ ciently.”
Generalized Few-shot Semantic Segmentation (2020, background): “Context information plays an important role for semantic segmentation and context modeling architectures are introduced, including global pooling [19] and pyramid pooling [5, 50, 63, 64].”
Lednet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation (2019, methodology): “To show the advantages of LEDNet, we selected 6 state-of-the-art lightweight networks as baselines, including SegNet [7], ENet [13], ERFNet [14], ICNet [19], CGNet [25], and ESPNet [20].”
Knowledge Distillation for Semantic Segmentation Using Channel and Spatial Correlations and Adaptive Cross Entropy (2020, methodology): “ICNet [16] proposed a method that divides the input images to low-, mid-, and high-resolution images using cascade feature fusion.”
Graph-Guided Architecture Search for Real-Time Semantic Segmentation (2019, methodology): “Compared to other methods such as SegNet [1], ENet [34], SQ [40] and ICNet [50], our method achieves signiﬁcant improvement in speed while getting performance gain over them about 14.8%, 13.5%, 12.0%, 2.3%, respectively.”
Is Context-Aware CNN Ready for the Surroundings? Panoramic Semantic Segmentation in the Wild (2021, background): “To list a few representative networks, ENet [31], SQNet [32], ERFNet [33], ICNet [16], LinkNet [34], ESPNet [35], EDANet [36], BiSeNet [37], CGNet [38] and SwiftNet [39].”