FasterSeg: Searching for Faster Real-time Semantic Segmentation

By Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang. 2019

209 citations recorded by the source corpus.

We present FasterSeg, an automatically designed semantic segmentation network with not only state-of-the-art performance but also faster speed than current methods. Utilizing neural architecture search (NAS), FasterSeg is discovered from a novel and broader search space integrating multi-resolution branches, that has been recently found to be vital in manually designed segmentation models. To better calibrate the balance between the goals of high accuracy and low latency, we propose a decoupled and fine-grained latency regularization, that effectively overcomes our observed phenomenons that the searched networks are prone to "collapsing" to low-latency yet poor-accuracy models. Moreover, we seamlessly extend FasterSeg to a new collaborative search (co-searching) framework, simultaneously searching for a teacher and a student network in the same single run. The teacher-student distillation further boosts the student model’s accuracy. Experiments on popular segmentation benchmarks demonstrate the competency of FasterSeg. For example, FasterSeg can run over 30% faster than the closest manually designed competitor on Cityscapes, while maintaining comparable accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhangyang Wang
2. Yunhe Wang
3. Xinggang Wang
4. Luc Van Gool
5. Kai Han
6. Chunhua Shen
7. Chunjing Xu
8. Qi Tian

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Chunhua Shen
4. Qi Tian
5. Yi Yang
6. Ming-Hsuan Yang
7. Xiaogang Wang
8. Zhangyang Wang

Citations in the corpus

• AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, positive): “During the search process, if we jointly update α and γ , we observe that our model is prone to suffering from the same “architecture collapse” problem observed in (Chen et al., 2019b), i.e., NAS is quickly biased towards some low-latency yet low-performance models.”
• AutoSpeech: Neural Architecture Search for Speaker Recognition (2020, unknown): “, image segmentation [20, 33], generative adversarial network [21], and phone recognition [19]), where task-specific search spaces are usually designed.”
• Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes (2023, methodology): “We use the protocols established by [58] for a fair comparison (image size: 2048 × 1024 for Cityscapes, 960 × 720 for CamVid, and 1184 × 896 for comma10k).”
• Rethinking BiSeNet For Real-time Semantic Segmentation (2021, background): “The prosperity of deep learning greatly promotes the performance of semantic segmentation by making various breakthroughs [18, 27, 22, 4], coming with fast-growing demands in many applications, e.”
• GhostNet: More Features From Cheap Operations (2019, methodology): “This kind of methods can also provide new search unit for automatic search methods [62, 55, 5].”
• Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, methodology): “On a separate line of research and to reduce the human efforts in designing neural networks, Neural Architecture Search (NAS) has mounted tremendous successes in various high-level vision tasks such as classification [13, 14, 15], object detection [16, 17], and semantic segmentation [18, 19, 20].”
• Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search (2020, methodology): “Recently, in one-shot NAS, there are few works leveraging this technique to boost the training of hypernetwork [ e.g., 42], and they commonly introduce additional large models as teachers.”
• TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation (2022, methodology): “AutoML techniques [5, 10, 34] are used to search for efficient architectures for scene parsing.”
• Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
• PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers (2022, methodology): “Using the measurement protocol proposed by [10] and following [20, 35, 45], we integrate the batch normalization into the convolutional layers and set the batch size to be 1 for measurement of inference speed.”
• OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Mapping (2022, methodology): “[10] Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, and Zhangyang Wang.”
• Improving Semantic Segmentation via Decoupled Body and Edge Supervision (2020, unknown): “Video Propagation [23]Wider-ResNetCityscapes 79.8 Ours ResNet-101 Cityscapes 81.5 Ours Wider-ResNetCityscapes 82.4 (a) Results on CamVid. Method Backbone mIoU(%) Dilated [26] ResNet101 55.2 FasterSeg [66] - 55.3 Ours ResNet101 66.9 (b) Results on BDD. Method IoU class(%)iIoU class(%)IoU category(%)i”
• CGNet: A Light-Weight Context Guided Network for Semantic Segmentation (2018, background): “is observed in the very recent FasterSeg [57].”
• Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search (2020, methodology): “Motivated by the promising performance achieved by those NAS works, recent works have extended NAS to more tasks such as segmentation [44, 10], image enhancement [20, 41], and language modeling [9].”
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (2021, background): “…interests, leading to principled and automated discovery for good architectures in a given search space of candidates (Zoph & Le, 2016; Brock et al., 2017; Pham et al., 2018; Liu et al., 2018a; Chen et al., 2018; Bender et al., 2018; Gong et al., 2019; Chen et al., 2020a; Fu et al., 2020).”