Fast and Practical Neural Architecture Search
By Jiequan Cui, Pengguang Chen, Ruiyu Li, Shu Liu, Xiaoyong Shen, Jiaya Jia. 2019
69 citations recorded by the source corpus.
In this paper, we propose a fast and practical neural architecture search (FPNAS) framework for automatic network design. FPNAS aims to discover extremely efficient networks with less than 300M FLOPs. Different from previous NAS methods, our approach searches for the whole network architecture to guarantee block diversity instead of stacking a set of similar blocks repeatedly. We model the search process as a bi-level optimization problem and propose an approximation solution. On CIFAR-10, our approach is capable of design networks with comparable performance to state-of-the-arts while using orders of magnitude less computational resource with only 20 GPU hours. Experimental results on ImageNet and ADE20K datasets further demonstrate transferability of the searched networks.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hanxiao Liu
- Xiaojun Chang
- Hangyu Zhu
- Jiaya Jia
- Yaochu Jin
- Pieter-Jan Kindermans
- Quoc V. Le
- Po-Yao (Bernie) Huang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Quoc V. Le
- Jiaya Jia
- Ming-Hsuan Yang
- Dacheng Tao
- Alan L. Yuille
- Chunhua Shen
- Xiaogang Wang
Citations in the corpus
- A Comprehensive Survey of Neural Architecture Search (2020, methodology): “5.0 224 - RL NASNet [31] CVPR18 82.7/96.2 88.9 331 2,000 NASNet-A [31] CVPR18 74.0/91.6 5.3 224 2,000 Block-QNN [32] CVPR18 77.4/93.5 N/A 224 96 Path-level EAS [56] ICML18 74.6/91.9 594 224 200 FPNAS [38] ICCV19 73.3/N/A 3.41 224 0.8 EA GeNet [16] ICCV17 72.1/90.4 156 224 17 Hierarchical-EAS [33] IC”
- Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS (2020, background): “In many of these methods, a single supernetwork is trained which encompasses all possible options in the search space [4, 28, 3, 5, 24, 41, 7].”
- Federated Learning on Non-IID Data: A Survey (2021, background): “• There is an increasing demand for automated machine learning (AutoML) and lots of algorithms have been presented on neural architecture search (NAS) in practical scenarios [22, 133].”
- Distilling Knowledge via Knowledge Review (2021, methodology): “There have been techniques for training fast and compact neural networks, including designing new architectures [10, 2, 11, 26], network pruning [20, 15, 34, 4, 19], quantization [13] , and knowledge distillation [9, 25].”
- Parametric Contrastive Learning (2021, methodology): “Note that, following [13], we use a weight-decay of 5e− 4.”