RENAS: Reinforced Evolutionary Neural Architecture Search

By Yukang Chen, Gaofeng Meng, Qian Zhang, Shiming Xiang, Chang Huang, Lisen Mu, Xinggang Wang. 2019

126 citations recorded by the source corpus.

Neural Architecture Search (NAS) is an important yet challenging task in network design due to its high computational consumption. To address this issue, we propose the Reinforced Evolutionary Neural Architecture Search (RENAS), which is an evolutionary method with reinforced mutation for NAS. Our method integrates reinforced mutation into an evolution algorithm for neural architecture exploration, in which a mutation controller is introduced to learn the effects of slight modifications and make mutation actions. The reinforced mutation controller guides the model population to evolve efficiently. Furthermore, as child models can inherit parameters from their parents during evolution, our method requires very limited computational resources. In experiments, we conduct the proposed search method on CIFAR-10 and obtain a powerful network architecture, RENASNet. This architecture achieves a competitive result on CIFAR-10. The explored network architecture is transferable to ImageNet and achieves a new state-of-the-art accuracy, i.e., 75.7% top-1 accuracy with 5.36M parameters on mobile ImageNet. We further test its performance on semantic segmentation with DeepLabv3 on the PASCAL VOC. RENASNet outperforms MobileNet-v1, MobileNet-v2 and NASNet. It achieves 75.83% mIOU without being pretrained on COCO.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Alan L. Yuille
2. Xiaodan Liang
3. Xiaojun Chang
4. Xia Hu
5. Cihang Xie
6. Song Bai
7. Qingquan Song
8. Wei David Liu

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Alan L. Yuille
3. Xiaogang Wang
4. Xiaodan Liang
5. Yi Yang
6. Dacheng Tao
7. Ming-Hsuan Yang
8. Trevor Darrell

Citations in the corpus

• AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search (2019, neutral): “2 3 4 6 [2, 3, 4, 6]”
• Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation (2019, unknown): “Early works [39, 3, 38, 7, 22] adopt an agent (e.”
• MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (2020, background): “Recently, many neural architecture search (NAS) methods have emerged to jointly learn the weights and the network architectures [4, 6, 12, 14, 22, 31, 34, 41, 44, 45, 71].”
• Neural Architecture Search for Lightweight Non-Local Networks (2020, methodology): “[8], we use the classification model as a drop-in replacement for the backbone feature extractor in the Deeplabv3 [6].”
• BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search (2021, methodology): “Without loss of generality, we consider searching with an evolutionary algorithm [11, 53], where architectures are optimized by evolving an architecture population {αp}.”
• Evolutionary Architecture Search for Graph Neural Networks (2020, methodology): “, image classification on CIFAR10/100 and ImageNet [13], [14].”
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction (2020, background): “gradient-based optimization [18], and tree-based methods [29]. Recent work has shown the effectiveness of combining different types of search techniques to better balance exploitation and exploration [5]. Due to the high complexity of training the DNNs and the particularity of evaluation criteria in”