PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search

By Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, H. Xiong. 2019

703 citations recorded by the source corpus.

Differentiable architecture search (DARTS) provided a fast solution in finding effective network architectures, but suffered from large memory and computing overheads in jointly training a super-net and searching for an optimal architecture. In this paper, we present a novel approach, namely Partially-Connected DARTS, by sampling a small part of super-net to reduce the redundancy in exploring the network space, thereby performing a more efficient search without comprising the performance. In particular, we perform operation search in a subset of channels while bypassing the held out part in a shortcut. This strategy may suffer from an undesired inconsistency on selecting the edges of super-net caused by sampling different channels. We solve it by introducing edge normalization, which adds a new set of edge-level hyper-parameters to reduce uncertainty in search. Thanks to the reduced memory cost, PC-DARTS can be trained with a larger batch size and, consequently, enjoy both faster speed and higher training stability. Experiment results demonstrate the effectiveness of the proposed method. Specifically, we achieve an error rate of 2.57% on CIFAR10 within merely 0.1 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.2% on ImageNet (under the mobile setting) within 3.8 GPU-days for search. Our code has been made available at https://www.dropbox.com/sh/on9lg3rpx1r6dkf/AABG5mt0sMHjnEJyoRnLEYW4a?dl=0.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Xiaodan Liang
  2. Quoc V. Le
  3. Qiang Yang
  4. Frank Hutter
  5. Yoshua Bengio
  6. Xiaojun Chang
  7. Colin White
  8. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Dacheng Tao
  2. Luc Van Gool
  3. Xiaodan Liang
  4. Yi Yang
  5. Alan L. Yuille
  6. Zhangyang Wang
  7. Ming-Hsuan Yang
  8. Chunhua Shen

In Efficiency & Compression

  1. Tien-Ju Yang
  2. Zhangyang Wang
  3. Vivienne Sze
  4. Chen Qian
  5. Fei Wang
  6. Shan You
  7. Wuyang Chen
  8. Xinyu Gong

In Neural Architecture Search & AutoML

  1. Colin White
  2. Yash Savani
  3. Arber Zela
  4. Frank Hutter
  5. Xiaojun Chang
  6. Julien N. Siems
  7. Willie Neiswanger
  8. Quanming Yao

Citations in the corpus