Customizable Architecture Search for Semantic Segmentation

By Yiheng Zhang, Zhaofan Qiu, Jingen Liu, Ting Yao, Dong Liu, Tao Mei. 2019

151 citations recorded by the source corpus.

In this paper, we propose a Customizable Architecture Search (CAS) approach to automatically generate a network architecture for semantic image segmentation. The generated network consists of a sequence of stacked computation cells. A computation cell is represented as a directed acyclic graph, in which each node is a hidden representation (i.e., feature map) and each edge is associated with an operation (e.g., convolution and pooling), which transforms data to a new layer. During the training, the CAS algorithm explores the search space for an optimized computation cell to build a network. The cells of the same type share one architecture but with different weights. In real applications, however, an optimization may need to be conducted under some constraints such as GPU time and model size. To this end, a cost corresponding to the constraint will be assigned to each operation. When an operation is selected during the search, its associated cost will be added to the objective. As a result, our CAS is able to search an optimized architecture with customized constraints. The approach has been thoroughly evaluated on Cityscapes and CamVid datasets, and demonstrates superior performance over several state-of-the-art techniques. More remarkably, our CAS achieves 72.3% mIoU on the Cityscapes dataset with speed of 108 FPS on an Nvidia TitanXp GPU.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

A Comprehensive Survey of Neural Architecture Search (2020, methodology): “With the improvement of NAS search efficiency, NAS is also quickly applied in the fields of object detection [65, 75, 111, 118], semantic segmentation [63, 64, 120], adversarial learning [53], architectural scaling [114, 122, 124], multi-objective optimization [39, 115, 125], platform-aware [28, 34,”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, methodology): “2019a; Zhang et al., 2019; Chen et al., 2019b) propose to use NAS to design backbone architectures speciﬁcally for segmentation or detection tasks.”
HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation (2022, background): “This concept was further improved in different aspects including increasing the receptive field while preserving spatial details [3,4,7,52,68,78,88], integrating context information [31,79,81,83,96], attention mechanisms [20,33,71,89], refining boundaries [17,37,82], neural architecture search [2, 4”
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation (2019, methodology): “Intersection over union (IoU) is used as the metric for comparison (mean ± s.d. %). neural architecture search, such an approach is hindered by the limited computational resources [19], [20], [21], [22], [23].”
Densely Connected Search Space for More Flexible Neural Architecture Search (2019, background): “CAS [50] customizes the search space design for real-time segmentation networks.”
FasterSeg: Searching for Faster Real-time Semantic Segmentation (2019, background): “1 CAS (Zhang et al., 2019) 71.”
Hierarchical Neural Architecture Search for Deep Stereo Matching (2020, background): “On a separate line of research and to reduce the human efforts in designing neural networks, Neural Architecture Search (NAS) has mounted tremendous successes in various high-level vision tasks such as classification [13, 14, 15], object detection [16, 17], and semantic segmentation [18, 19, 20].”
AdversarialNAS: Adversarial Neural Architecture Search for GANs (2019, methodology): “The NAS technique has applied to many computer vision tasks such as image classiﬁcation [2, 28, 29, 34, 52, 4], dense image prediction [27, 50, 3] and object detection [11, 33].”
Lite-HRNet: A Lightweight High-Resolution Network (2021, methodology): “02 GFLOPs, outperforming the hand-crafted methods [34, 55, 50, 23, 58, 36, 41, 12] and NAS-based methods [54, 25, 16, 24] , and comparable with SwiftNetRN18 [32] that is far computationally intensive (104 GFLOPs).”
Semantic Flow for Fast and Accurate Scene Parsing (2020, methodology): “…69.5 34 26.5M DF1-Seg [48] 1024 × 2048 73.0 80 8.55M DF2-Seg [48] 1024 × 2048 74.8 55 8.55M SwiftNet [38] 1024 × 2048 75.5 39.9 11.80M SwiftNet-ens [38] 1024 × 2048 76.5 18.4 24.7M DFANet [23] 1024 × 1024 71.3 100 7.8M CellNet [58] 768 CamVid: is another road scene dataset for autonomous driving.”
MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (2020, methodology): “This is realized by various search algorithms including Bayesian optimization [3], reinforcement learning [18, 58, 72], evolutionary algorithms [49, 63], network transformation [11, 17], and gradient descent [1, 33, 61, 70].”
Searching Central Difference Convolutional Networks for Face Anti-Spoofing (2020, methodology): “For the perspective of computer vision applications, NAS has been developed for face recognition [67], action recognition [46], person ReID [50], object detection [21] and segmentation [65] tasks.”