Auto-FPN: Automatic Network Architecture Adaptation for Object Detection Beyond Classification

By Hang Xu, Lewei Yao, Zhenguo Li, Xiaodan Liang, Wei Zhang. 2019

199 citations recorded by the source corpus.

Abstract Neural architecture search (NAS) has shown great potential in automating the manual process of designing a good CNN architecture for image classification. In this paper, we study NAS for object detection, a core computer vision task that classifies and localizes object instances in an image. Existing works focus on transferring the searched architecture from classification task (ImageNet) to the detector backbone, while the rest of the architecture of the detector remains unchanged. However, this pipeline is not task-specific or data-oriented network search which cannot guarantee optimal adaptation to any dataset. Therefore, we propose an architecture search framework named Auto-FPN specifically designed for detection beyond simply searching a classification backbone. Specifically, we propose two auto search modules for detection: Auto-fusion to search a better fusion of the multi-level features; Auto-head to search a better structure for classification and bounding-box(bbox) regression. Instead of searching for one repeatable cell structure, we relax the constraint and allow different cells. The search space of both modules covers many popular designs of detectors and allows efficient gradient-based architecture search with resource constraint (2 days for COCO on 8 GPU cards). Extensive experiments on Pascal VOC, COCO, BDD, VisualGenome and ADE demonstrate the effectiveness of the proposed method, e.g. achieving around 5% improvement than FPN in terms of mAP while requiring around 50% fewer parameters on the searched modules.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neural Architecture Search & AutoML

Citations in the corpus

AutoML: A Survey of the State-of-the-Art (2019, methodology): “Therefore, NAS has also been applied to many other computer vision tasks, such as object detection [153, 154, 155], semantic segmentation [156, 157], image restoration [158], GAN [159], graph neural network (GNN) [160].”
VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, methodology): “Note that NAS and data augmentation are cutting-edge deep learning techniques and proved to be effective in applications (Xie et al., 2020; Yao et al., 2019; Xu et al., 2019; 2020).”
DARTS+: Improved Differentiable Architecture Search with Early Stopping (2019, methodology): “The neural architectures searched by NAS have achieved the state-of-the-art results over handcrafted neural architectures in various tasks, including object classiﬁcation [4, 20, 21, 25, 26, 40], object detection [7, 36], semantic segmentation [19], recommender systems [14], etc.”
Deformable DETR: Deformable Transformers for End-to-End Object Detection (2020, methodology): “Recently, NAS-FPN (Ghiasi et al., 2019) and Auto-FPN (Xu et al., 2019) are proposed to automatically design cross-scale connections via neural architecture search.”
Dilated SpineNet for Semantic Segmentation (2021, background): “Recent architectures discovered by NAS have shown promising results than handcrafted models on vision tasks including image classification [57, 31, 41, 20], object detection [16, 14, 13, 23, 43, 46, 24], semantic segmentation [27, 34], etc.”
Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation (2020, unknown): “In this paper, inspired by the advances achieved by Network Architecture Search (NAS) on fundamental vision tasks [27, 2, 36, 41], we aim to explore an efficient foreground-background cooperative architecture automatically tailored for panoptic segmentation, namely Auto-Panoptic.”
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution (2020, methodology): “NAS-FPN [24] and Auto-FPN [73] use neural architecture search [87] to ﬁnd the optimal FPN structure.”
Revisiting the Sibling Head in Object Detector (2020, background): “Even the advanced algorithms bolstered by AutoML [8, 38] have been delved, the performance gain is still limited to an easily accessible improvement range.”
Graph-Guided Architecture Search for Real-Time Semantic Segmentation (2019, methodology): “Recently, NAS also has been used in object detection, such as NAS-FPN [16], DetNAS [48] and Auto-FPN [45].”
Densely Connected Search Space for More Flexible Neural Architecture Search (2019, background): “…years, neural architecture search (NAS) [52, 53, 37, 39] has demonstrated great successes in designing neural architectures automatically and achieved remarkable performance gains in various tasks such as image classiﬁcation [53, 37], semantic segmentation [6, 28] and object detection [17, 47].”
CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending (2020, background): “Most works search a basic CNN architectures for a classification model [18, 3, 16, 26, 29] while a few of them focus on more complicated high-level vision tasks such as semantic segmentation and object detection [4, 5, 30, 31].”
MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (2020, methodology): “NAS methods have produced highly competitive architectures on various computer vision tasks, such as image classification [5, 8, 9, 25, 35, 43, 59, 71], object detection [7, 14, 66], and semantic segmentation [33, 44].”
SP-NAS: Serial-to-Parallel Backbone Search for Object Detection (2020, methodology): “Table 5 shows the searched SPNet BDD is faster than Auto-FPN [46] with better performance.”
D2Det: Towards High Quality Object Detection and Instance Segmentation (2020, unknown): “3 Auto-FPN [49] ResNet101 ∼ 1333× 800 42.”