RC-DARTS: Resource Constrained Differentiable Architecture Search

By Xiaojie Jin, Jiang Wang, Joshua Slocum, Ming-Hsuan Yang, Shengyang Dai, Shuicheng Yan, Jiashi Feng. 2019

37 citations recorded by the source corpus.

Recent advances show that Neural Architectural Search (NAS) method is able to find state-of-the-art image classification deep architectures. In this paper, we consider the one-shot NAS problem for resource constrained applications. This problem is of great interest because it is critical to choose different architectures according to task complexity when the resource is constrained. Previous techniques are either too slow for one-shot learning or does not take the resource constraint into consideration. In this paper, we propose the resource constrained differentiable architecture search (RC-DARTS) method to learn architectures that are significantly smaller and faster while achieving comparable accuracy. Specifically, we propose to formulate the RC-DARTS task as a constrained optimization problem by adding the resource constraint. An iterative projection method is proposed to solve the given constrained optimization problem. We also propose a multi-level search strategy to enable layers at different depths to adaptively learn different types of neural architectures. Through extensive experiments on the Cifar10 and ImageNet datasets, we show that the RC-DARTS method learns lightweight neural architectures which have smaller model size and lower computational complexity while achieving comparable or better performances than the state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Alan L. Yuille
2. Yingwei Li
3. Xi Li
4. Jianchao Yang
5. Xiaojie Jin
6. Linjie Yang
7. Jieru Mei
8. Yongjian Fu

In Computer Vision and Pattern Recognition

1. Xi Li
2. Alan L. Yuille
3. Luc Van Gool
4. Ming-Hsuan Yang
5. Dacheng Tao
6. Yi Yang
7. Xiaogang Wang
8. Jiashi Feng

Citations in the corpus

• AtomNAS: Fine-Grained End-to-End Neural Architecture Search (2019, background): “The similar idea of optimizing optimal subgraphs within a supergraph is also adopted by Liu et al. (2019a); Jin et al.”
• What and Where: Learn to Plug Adapters via NAS for Multidomain Learning (2020, background): “There is a rich body of works in NAS, which are mainly based on three strategies: reinforcement learning method [21]–[24], evolutionary algorithm [25] and gradient-based optimization [26]–[30].”