Searching for Efficient Multi-Scale Architectures for Dense Image Prediction

By Liang-Chieh Chen, Maxwell D. Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, Jonathon Shlens. 2018

425 citations recorded by the source corpus.

The design of neural network architectures is an important component for achieving state-of-the-art performance with machine learning systems across a broad array of tasks. Much work has endeavored to design and build architectures automatically through clever construction of a search space paired with simple learning algorithms. Recent progress has demonstrated that such meta-learning methods may exceed scalable human-invented architectures on image classification tasks. An open question is the degree to which such methods may generalize to new domains. In this work we explore the construction of meta-learning techniques for dense image prediction focused on the tasks of scene parsing, person-part segmentation, and semantic image segmentation. Constructing viable search spaces in this domain is challenging because of the multi-scale representation of visual information and the necessity to operate on high resolution imagery. Based on a survey of techniques in dense image prediction, we construct a recursive search space and demonstrate that even with efficient random search, we can identify architectures that outperform human-invented architectures and achieve state-of-the-art performance on three dense prediction tasks including 82.7% on Cityscapes (street scene parsing), 71.3% on PASCAL-Person-Part (person-part segmentation), and 87.9% on PASCAL VOC 2012 (semantic image segmentation). Additionally, the resulting architecture is more computationally efficient, requiring half the parameters and half the computational cost as previous state of the art systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “Some employ atrous convolu-tions [32,72,62,12], larger kernel [65], or image pyramids [92,80], either designed by hand or searched by algorithms [97,11,55].”
Stand-Alone Self-Attention in Vision Models (2019, background): “The field of deep learning has in turn largely shifted toward the design of architectures of CNNs for improving the performance on image recognition [11–16], object detection [17–19] and image segmentation [20–22].”
CCNet: Criss-Cross Attention for Semantic Segmentation (2018, methodology): “BiSeNet [35] was designed for real-time semantic segmentation.”
Randaugment: Practical automated data augmentation with a reduced search space (2019, background): “This formulation makes a strong assumption that the proxy task provides a predictive indication of the larger task [28, 2].”
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “Already by now, NAS methods have outperformed manually designed architectures on a range of tasks such as image classification (Tan & Le, 2019; Tan et al., 2019; Howard et al., 2019; Liu et al., 2018a), and segmentation (Chen et al., 2018a; Liu et al., 2019; Chen et al., 2019b).”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, methodology): “Compared to NAS methods, FNA costs 1737 × less than DPC (Chen et al., 2018a), 6.8 × less than Auto-DeepLab (Liu et al., 2019a) and 7.4 × less than DetNAS (Chen et al., 2019b).”
AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, methodology): “Recent works further extended the NAS algorithms to dense and structured prediction [7, 36].”
Searching for a Robust Neural Architecture in Four GPU Hours (2019, background): “Recently, neural architecture search (NAS) has increasingly attracted the interest of researchers [1, 6, 17, 22, 25, 31, 46].”
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (2019, methodology): “Beyond image classification, Neural Architecture Search has also been used to improve image segmentation networks [4].”
NAS-Unet: Neural Architecture Search for Medical Image Segmentation (2019, background): “Some work has been trying to solve the above two problems and achieved some success — exactly the idea that recent work has followed [24], [25].”
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, methodology): “…[31] - - Cls DeepLabv3+ [11] - - Seg NASNet [93] CIFAR-10 2000 Cls AmoebaNet [62] CIFAR-10 2000 Cls PNASNet [47] CIFAR-10 150 Cls DARTS [49] CIFAR-10 4 Cls DPC [6] Cityscapes 2600 Seg Auto-DeepLab Cityscapes 3 Seg Table 1: Comparing our work against other CNN architectures with two-level hierarchy.”
Object-Contextual Representations for Semantic Segmentation (2019, neutral): “1 - - - DPC [4] Xception-71 4× M 82.”
NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search (2018, methodology): “Chen et al. show in Chen et al. (2018a) that an augmented random search approach to optimize networks for a semantic segmentation application.”