Practical Network Blocks Design with Q-Learning

By Zhaobai Zhong, Junjie Yan, Chenglin Liu. 2017

127 citations recorded by the source corpus.

Convolutional neural network provides an end-to-end solution to train many computer vision tasks and has gained great successes. However, the design of network architectures usually relies heavily on expert knowledge and is hand-crafted. In this paper, we provide a solution to automatically and efficiently design high performance network architectures. To reduce the search space of network design, we focus on constructing network blocks, which can be stacked to generate the whole network. Blocks are generated through an agent, which is trained with Q-learning to maximize the expected accuracy of the searching blocks on the learning task. Distributed asynchronous framework and early stop strategy are used to accelerate the training process. Our experimental results demonstrate that the network architectures designed by our approach perform competitively compared with hand-crafted state-of-the-art networks. We trained the Q-learning on CIFAR-100, and evaluated on CIFAR10 and ImageNet, the designed block structure achieved 3.60% error on CIFAR-10 and competitive result on ImageNet. The Q-learning process can be efficiently trained only on 32 GPUs in 3 days.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neurology, Reinforcement Learning, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neurology

In Reinforcement Learning

In Neural Architecture Search & AutoML

Citations in the corpus

Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution (2018, methodology): “Neural Architecture Search Recently, NAS was framed as a reinforcement learning (RL) problem, where the reward of the RL agent is based on the validation performance of the trained architecture (Baker et al., 2017a; Zoph & Le, 2017; Zhong et al., 2018; Pham et al., 2018).”
Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor (2020, background): “In this subsection, ) [54] 93.57 74.84 – ResNet (depth=1,202) [54] 92.07 72.18 – Maxout [57] 90.70 61.40 – VGG [58] 93.34 7.95 – Network in Network [59] 91.19 64.32 – Highway Network [60] 92.28 67.61 – All-CNN [61] 92.75 66.29 – FractalNet [62] 94.78 77.70 – CNN architecture design algorithms based ”
Adversarial AutoAugment (2019, methodology): “Inspired by neural architecture search (NAS)(Zoph & Le, 2016; Zoph et al., 2017; Zhong et al., 2018a;b; Guo et al., 2018), a reinforcement learning (RL) (Williams, 1992) method called AutoAugment is proposed by Cubuk et al. (2018), which can automatically learn the augmentation policy from data and…”
Efficient Progressive Neural Architecture Search (2018, methodology): “This is a choice that is common in the literature [15, 18, 29, 31].”
Auto-Keras: An Efficient Neural Architecture Search System (2018, background): “Many NAS approaches, such as deep reinforcement learning [2, 30, 40, 41], gradient-based methods [26] and evolutionary algorithms [10, 23, 31, 32, 34], require a large n to reach a good”
Path-Level Network Transformation for Efficient Architecture Search (2018, methodology): “, 2018) Block-QNN-A (N = 4) (Zhong et al., 2017) NAS v3 (Zoph & Le, 2017) NASNet-A (6, 32) + DropPath (600 epochs) (Zoph et al.”
Transfer Learning with Neural AutoML (2018, background): “For example, Neural Architecture Search has discovered novel networks that rival the best human-designed architectures on challenging image classification tasks [9, 10].”
Neural Architecture Search with Bayesian Optimisation and Optimal Transport (2018, neutral): “[49] Zhao Zhong, Junjie Yan, and Cheng-Lin Liu.”
AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “Following [57, 4, 54], we use the following reward:”
Peephole: Predicting Network Performance Before Training (2017, methodology): “[26] also proposed to search transferable blocks (referred to as cells in [28]) and assemble them into a network in their efforts towards an automatic way for network search.”
A genetic programming approach to designing convolutional neural network architectures (2017, background): “Since these approaches require much computational cost in general, the recent works concentrate on the reduction of computational cost and improvement of architecture search efficiency [Cai et al., 2018; Liu et al., 2017; 2018; Pham et al., 2018; Real et al., 2018; Zhong et al., 2017; Zoph et al., 2”
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction (2018, methodology): “Many parallel efforts have employed reinforcement learning [3, 99], evolutionary algorithms [81, 69, 59, 90, 53, 68] and sequential model-based optimization [61, 52] to learn network structures.”
NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search (2018, methodology): “State-of-the-art reinforcement learning (RL) Baker et al. (2017); Zhong et al. (2017); Zoph & Le (2016); Zoph et al. (2017); Hsu et al. (2018a); Cai et al. (2018); Pham et al. (2018) and evolutionary algorithm (EA) Miikkulainen et al. (2017); Real et al. (2017; 2018); Xie & Yuille (2017); Kim et al.”
DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures (2018, methodology): “Other works in the field including approach used in [22] which searches model architectures by manipulating the depth of the width of the layers using policy gradient, and the methods proposed by [8] and [10] which search network architectures using Q-learning.”
Demystifying Parallel and Distributed Deep Learning (2018, methodology): “ First-Order[43,123,126,141,145,163,227] Second-Order[11,28,56,94,133,165,172] Evolutionary[169,192,234,242] Hyper-Parameter Search[13,89,91,118,131,163,169,219,256] Architecture Search: Reinforcement[12,193,264,268,269], Evolutionary[160,203,204,242,251], SMBO[27,71,159,161,176] Fig. 18. Overview o”
Exploiting the Potential of Standard Convolutional Autoencoders for Image Restoration by Evolutionary Search (2018, methodology): “There are studies that employ the REINFORCE algorithm, policy gradient, and Q-learning to learn network topology (Zoph & Le, 2017; Baker et al., 2017; Zhong et al., 2017; Zoph et al., 2017).”
Network Transplanting (2018, background): “Similarly, some studies have explored new representation structures instead of neural networks, such as forests and decision trees [12, 32, 7, 25] and automatic learning of optimal network structures [33, 14, 34, 31].”
Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS (2019, background): “Many kinds of research in the ﬁeld of so-called automated machine learning (Auto-ML) have been made (Zhong et al. (2017); Cai et al. (2018)).”