BlockQNN: Efficient Block-Wise Neural Network Architecture Generation

By Zhaobai Zhong, Zichen Yang, Boyang Deng, Junjie Yan, Wei Wu, Jing Shao, Chenglin Liu. 2018

137 citations recorded by the source corpus.

Convolutional neural networks have gained a remarkable success in computer vision. However, most popular network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained to choose component layers sequentially. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it yields state-of-the-art results in comparison to the hand-crafted networks on image classification, particularly, the best network generated by BlockQNN achieves 2.35 percent top-1 error rate on CIFAR-10. (2) it offers tremendous reduction of the search space in designing networks, spending only 3 days with 32 GPUs. A faster version can yield a comparable result with only 1 GPU in 20 hours. (3) it has strong generalizability in that the network built on CIFAR also performs well on the larger-scale dataset. The best network achieves very competitive accuracy of 82.0 percent top-1 and 96.0 percent top-5 on ImageNet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Adversarial AutoAugment (2019, methodology): “Inspired by neural architecture search (NAS)(Zoph & Le, 2016; Zoph et al., 2017; Zhong et al., 2018a;b; Guo et al., 2018), a reinforcement learning (RL) (Williams, 1992) method called AutoAugment is proposed by Cubuk et al. (2018), which can automatically learn the augmentation policy from data and…”
Single Path One-Shot Neural Architecture Search with Uniform Sampling (2019, methodology): “Note that this is different from either the nested optimization [37, 38, 34, 35, 1] or the joint optimization [26, 4, 15, 29, 32].”
A review of convolutional neural network architectures and their optimizations (2022, methodology): “2020a), reinforcement learning (Zoph and Le 2016; Tan et al. 2019; Zhong et al. 2020; Cheng et al. 2020), gradient descent-based method (Chen and Hsieh 2020; Chu et al.”
AGAN: Towards Automated Design of Generative Adversarial Networks (2019, methodology): “We point out that it is at least in contrast to RL-based NAS algorithms over image classifiers [25, 27, 13], where both the normal cell and reduction cell choose among depthwise-separable convolutions, max and average poolings.”