Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation

By Xin Chen, Lingxi Xie, Jun Wu, Qi Tian. 2019

738 citations recorded by the source corpus.

Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdarts

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Dacheng Tao
  3. Mohit Bansal
  4. Yoshua Bengio
  5. Jianfeng Gao
  6. Tie-Yan Liu
  7. Alan L. Yuille
  8. Trevor Darrell

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Xiaogang Wang
  4. Ming-Hsuan Yang
  5. Yi Yang
  6. Alan L. Yuille
  7. Trevor Darrell
  8. Chunhua Shen

In Neural Architecture Search & AutoML

  1. Quoc V. Le
  2. Xiaojun Chang
  3. Bo Zhang
  4. Xiangxiang Chu
  5. Chaoyang He
  6. Qi Tian
  7. Zhangyang Wang
  8. Xin Chen

Citations in the corpus