Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells

By Vladimir Nekrasov, Hao Chen, Chunhua Shen, Ian David Reid. 2018

155 citations recorded by the source corpus.

Automated design of neural network architectures tailored for a specific task is an extremely promising, albeit inherently difficult, avenue to explore. While most results in this domain have been achieved on image classification and language modelling problems, here we concentrate on dense per-pixel tasks, in particular, semantic image segmentation using fully convolutional networks. In contrast to the aforementioned areas, the design choices of a fully convolutional network require several changes, ranging from the sort of operations that need to be used - e.g., dilated convolutions - to a solving of a more difficult optimisation problem. In this work, we are particularly interested in searching for high-performance compact segmentation architectures, able to run in real-time using limited resources. To achieve that, we intentionally over-parameterise the architecture during the training time via a set of auxiliary cells that provide an intermediate supervisory signal and can be omitted during the evaluation phase. The design of the auxiliary cell is emitted by a controller, a neural network with the fixed structure trained using reinforcement learning. More crucially, we demonstrate how to efficiently search for these architectures within limited time and computational budgets. In particular, we rely on a progressive strategy that terminates non-promising architectures from being further trained, and on Polyak averaging coupled with knowledge distillation to speed-up the convergence. Quantitatively, in 8 GPU-days our approach discovers a set of architectures performing on-par with state-of-the-art among compact models on the semantic segmentation, pose estimation and depth prediction tasks. Code will be made available here: https://github.com/drsleep/nas-segm-pytorch

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neural Architecture Search & AutoML

Citations in the corpus

VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, neutral): “Adelaide-EA (Nekrasov et al., 2019).”
A Comprehensive Survey of Neural Architecture Search (2020, methodology): “twork architecture [18–20, 48, 49, 52, 84, 105]. With the improvement of NAS search efficiency, NAS is also quickly applied in the fields of object detection [65, 75, 111, 118], semantic segmentation [63, 64, 120], adversarial learning [53], architectural scaling [114, 122, 124], multi-objective opt”
Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising (2019, methodology): “Architectures automatically found by algorithms have achieved highly competitive performance in high-level vision tasks such as image classiﬁcation [46], object detection [8, 36] and semantic segmentation [17, 29].”
DetNAS: Backbone Search for Object Detection (2019, background): “[18] proposes to search auxiliary cells as the segmentation decoders.”
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021, background): “29 ]; introducing boundary information [30–37]; designing various attention modules [38–46]; or using AutoML technologies [47–51].”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
Graph-Guided Architecture Search for Real-Time Semantic Segmentation (2019, background): “And [31] over-parameterises the architecture during the training via a set of auxiliary cells using reinforcement learning.”
Learning Dynamic Routing for Semantic Segmentation (2020, methodology): “With the development of NAS, there are several methods attempting to search for a static network automatically [3, 22, 25].”
NAS-FCOS: Fast Neural Architecture Search for Object Detection (2019, methodology): “On the other hand, Neural Architecture Search (NAS) approaches [4,14,26] have been showing impressive results on automatically discovering top-performing neural network architectures in large-scale search spaces.”
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search (2020, methodology): “NAS algorithms have been successfully applied to various tasks, including image classification [53, 61, 63, 64, 89, 90], semantic segmentation [14,51,58], object detection [24], image restoration [17,72], and image generation [26].”
IR-NAS: Neural Architecture Search for Image Restoration (2019, background): “Architectures found automatically have achieved highly competitive performance in high-level vision tasks such as image classification [1], object detection [2] and semantic segmentation [3, 4].”
HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking (2019, background): “Neural architecture search (NAS) has recently attracted significant interests due to its capability of automating neural network architecture design and its success in outperforming hand-crafted architectures in many important tasks such as image classification [1], object detection [2], and semanti”
DetNAS: Neural Architecture Search on Object Detection (2019, background): “Some [16] proposes to search auxiliary cells as the segmentation decoders.”
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation (2022, methodology): “AutoML techniques [5, 10, 34] are used to search for efficient architectures for scene parsing.”
FasterSeg: Searching for Faster Real-time Semantic Segmentation (2019, methodology): “5), that aims to transfer learned knowledge from a sophisticated teacher network to a light-weight student, to improve the (more efﬁcient) student’s accuracy. For segmentation, Liu et al. (2019b) and Nekrasov et al. (2019) proposed to leverage knowledge distillation to improve the accuracy of the co”
Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation (2020, unknown): “Neural architecture search (NAS) aims at automating the labor-intensive architecture engineering process and has shown promising results on fundamental computer vision tasks like image classiﬁcation [1, 27, 39], object detection [32, 36, 13] and semantic segmentation [26, 31].”