Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation

By Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan L. Yuille, Li Fei-Fei. 2019

1,052 citations recorded by the source corpus.

Recently, Neural Architecture Search (NAS) has successfully identified neural network architectures that exceed human designed ones on large-scale image classification. In this paper, we study NAS for semantic image segmentation. Existing works often focus on searching the repeatable cell structure, while hand-designing the outer network structure that controls the spatial resolution changes. This choice simplifies the search space, but becomes increasingly problematic for dense image prediction which exhibits a lot more network level architectural variations. Therefore, we propose to search the network level structure in addition to the cell level structure, which forms a hierarchical architecture search space. We present a network level search space that includes many popular designs, and develop a formulation that allows efficient gradient-based architecture search (3 P100 GPU days on Cityscapes images). We demonstrate the effectiveness of the proposed method on the challenging Cityscapes, PASCAL VOC 2012, and ADE20K datasets. Auto-DeepLab, our architecture searched specifically for semantic image segmentation, attains state-of-the-art performance without any ImageNet pretraining.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “…with 37.8% fewer parameters and 35% fewer M-Adds, performs better than the Panoptic-DeepLab (Auto-DeepLab-XL++), not only on panoptic segmentation (0.8% PQ) and instance segmentation (0.3% AP), but also on semantic segmentation (0.8% mIoU), the task that Auto-DeepLab [55] was searched for.”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, background): “3, our decoder is stacked L times, traversing output strides (4, 8, and 16 [16, 61]) multiple times.”
AutoML: A Survey of the State-of-the-Art (2019, neutral): “Besides, some works are proposed to solve the task of object detection [140] and semantic segmentation [141], [142].”
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “Already by now, NAS methods have outperformed manually designed architectures on a range of tasks such as image classification (Tan & Le, 2019; Tan et al., 2019; Howard et al., 2019; Liu et al., 2018a), and segmentation (Chen et al., 2018a; Liu et al., 2019; Chen et al., 2019b).”
A Comprehensive Survey of Neural Architecture Search (2020, methodology): “With the improvement of NAS search efficiency, NAS is also quickly applied in the fields of object detection [65, 75, 111, 118], semantic segmentation [63, 64, 120], adversarial learning [53], architectural scaling [114, 122, 124], multi-objective optimization [39, 115, 125], platform-aware [28, 34,”
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch (2020, background): “In only two years, for example, comparable experiments went from requiring hundreds of GPUs (Zoph et al., 2018) to only one (Liu et al., 2019b).”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, methodology): “For fair comparison with Auto-DeepLab (Liu et al., 2019a) which searches the back-bone architecture on DeepLabv3 and retrains the searched network on DeepLabv3+ (Chen et al., 2018b), we adapt the parameters of the target architecture Arch t to DeepLabv3+ framework.”
Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation (2019, background): “Diﬀerent approaches [16,8,9,17,12,7] have been introduced in computer vision to search for the best neural network architecture for image analysis and scene understanding tasks.”
Understanding and Robustifying Differentiable Architecture Search (2019, methodology): “This might arise potentially from some premature convergence in the weights’ space, supposing that the landscape of the training loss L train does not change signiﬁcantly after each α update.”
DARTS+: Improved Differentiable Architecture Search with Early Stopping (2019, methodology): “The neural architectures searched by NAS have achieved the state-of-the-art results over handcrafted neural architectures in various tasks, including object classiﬁcation [4, 20, 21, 25, 26, 40], object detection [7, 36], semantic segmentation [19], recommender systems [14], etc.”
Auto-GNN: Neural architecture search of graph neural networks (2019, methodology): “The deep architectures discovered by NAS algorithms have outperformed the handcrafted ones at the domains including image classification [12–21], semantic image segmentation [22], and image generation [23].”
AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “Neural architecture search (NAS) has been successfully developed and evaluated on the task of image classiﬁcation [71, 46], and lately on image segmentation as well [7, 36].”
Video Action Recognition Via Neural Architecture Searching (2019, methodology): “In semantic image segmentation, AutoDeeplab [12] also performs well even without a pre-train procedure.”
BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, methodology): “Lastly, soij should work for both proxy and proxyless scenarios and can be better embedded into existing algorithmic frameworks (Liu et al., 2019b; Cai et al., 2019; Gordon et al., 2018).”
A Survey on Neural Architecture Search (2019, methodology): “…other dimensions that address more complex tasks with image data such as object detection and segmentation (Zoph et al., 2018; Tan et al., 2018; Liu et al., 2019a; Weng et al., 2019), as well as works that address safety-critical issues such as the discovery of architectures that are robust…”
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation (2019, background): “Recently, the development of neural architecture search (NAS) has changed the convention of model design from manual to automatic, achieving remarkable success in various perceptual tasks [16, 3, 37] including image recognition [38].”