Practical Block-Wise Neural Network Architecture Generation

By Zhaobai Zhong, Junjie Yan, Wei Wu, Jing Shao, Chenglin Liu. 2017

531 citations recorded by the source corpus.

Convolutional neural networks have gained a remarkable success in computer vision. However, most usable network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained sequentially to choose component layers. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it performs competitive results in comparison to the hand-crafted state-of-the-art networks on image classification, additionally, the best network generated by BlockQNN achieves 3.54% top-1 error rate on CIFAR-10 which beats all existing auto-generate networks. (2) in the meanwhile, it offers tremendous reduction of the search space in designing networks which only spends 3 days with 32 GPUs, and (3) moreover, it has strong generalizability that the network built on CIFAR also performs well on a larger-scale ImageNet dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neurology, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Dacheng Tao
  3. Mohit Bansal
  4. Yoshua Bengio
  5. Jianfeng Gao
  6. Song Han
  7. Sergey Levine
  8. Percy Liang

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Xiaogang Wang
  5. Yi Yang
  6. Chunhua Shen
  7. Trevor Darrell
  8. Alan L. Yuille

In Neurology

  1. Song Han
  2. Martin Wistuba
  3. Bing Xue
  4. Mengjie Zhang
  5. Yanan Sun
  6. Gary G. Yen
  7. Hanxiao Liu
  8. Karen Simonyan

In Neural Architecture Search & AutoML

  1. Martin Wistuba
  2. Song Han
  3. Frank Hutter
  4. Quoc V. Le
  5. Tejaswini Pedapati
  6. Thomas Elsken
  7. Yanan Sun
  8. Bing Xue

Citations in the corpus