Reinforcement Learning for Architecture Search by Network Transformation

By Han Cai, Tianyao Chen, Weinan Zhang, Yong Yu, Jun Wang. 2017

67 citations recorded by the source corpus.

Deep neural networks have shown effectiveness in many challenging tasks and proved their strong capability in automatically learning good feature representation from raw input. Nonetheless, designing their architectures still requires much human effort. Techniques for automatically designing neural network architectures such as reinforcement learning based approaches recently show promising results in benchmarks. However, these methods still train each network from scratch during exploring the architecture space, which results in extremely high computational cost. In this paper, we propose a novel reinforcement learning framework for automatic architecture designing, where the action is to grow the network depth or layer width based on the current network architecture with function preserved. As such, the previously validated networks can be reused for further exploration, thus saves a large amount of computational cost. The experiments on image benchmark datasets have demonstrated the efficiency and effectiveness of our proposed solution compared to existing automatic architecture designing methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Quoc V. Le
2. Song Han
3. Tie-Yan Liu
4. Frank Hutter
5. Tao Qin
6. Xia Hu
7. Mingxing Tan
8. Qingquan Song

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Quoc V. Le
3. Song Han
4. Dacheng Tao
5. Alan L. Yuille
6. Yi Yang
7. Trevor Darrell
8. Xiaogang Wang

In Reinforcement Learning

1. Fei Tian
2. Tao Qin
3. Tie-Yan Liu
4. Mingxing Tan
5. Song Han
6. Catherine Wong
7. Frank Hutter
8. Thomas Brox

In Neural Architecture Search & AutoML

1. Frank Hutter
2. Xia Hu
3. Qingquan Song
4. Arber Zela
5. Quoc V. Le
6. Martin Wistuba
7. Song Han
8. Thomas Brox

Citations in the corpus

• Neural Architecture Optimization (2018, methodology): “Automatic design of neural network architecture without human intervention has been the interests of the community from decades ago [12, 22] to very recent [47, 48, 27, 39, 7].”
• Auto-GNN: Neural architecture search of graph neural networks (2019, background): “Most of NAS frameworks are built up based on one of the two basic algorithms: RL [12, 13, 15, 38, 39] and EA [14, 40–43].”
• EENA: Efficient Evolution of Neural Architecture (2019, background): “There are several precedents [1, 35, 2] based on this for neural architecture search, but the basic operations are limited to the experience in parameters and are relatively simple, so the algorithms may degenerate into a random search.”
• Transfer Learning with Neural AutoML (2018, background): “[23] transform existing architectures incrementally to avoid generating entire networks from scratch.”
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, methodology): “During the update, the baseline reward b is subtracted to reduce the variance of gradient estimation, which is an exponential moving average of the previous rewards [56, 6]:”
• Simple And Efficient Architecture Search for Convolutional Neural Networks (2017, methodology): “This led to a growing interest in using evolutionary algorithms (Real et al., 2017; Suganuma et al., 2017) and reinforcement learning (Baker et al., 2016; Cai et al., 2017; Zoph & Le, 2017) for automatically designing CNN architectures.”
• Searching for Activation Functions (2018, unknown): “Zoph et al., 2017; Real et al., 2017; Cai et al., 2017; Zhong et al., 2017) and optimizers (Bello et al., 2017).”
• MnasNet: Platform-Aware Neural Architecture Search for Mobile (2018, methodology): “hitecture search has been further developed, with progressive search methods (Liu et al. 2018a), parameter sharing (Pham et al. 2018), hierarchical search spaces (Liu et al. 2018b), network transfer (Cai et al. 2018), evolutionary search (Real et al. 2018), or differentiable search algorithms (Liu, ”
• AutoDispNet: Improving Disparity Estimation With AutoML (2019, background): “The same strategy is adopted in many follow-up works [6, 13, 14, 69, 78, 79, 82].”
• Convolutional Neural Networks (2018, methodology): “As mentioned above, Cai et al. (2017) used network morphisms for architecture search, though they just employ the Net2Deeper and Net2Wider operators from Chen et al.”
• Understanding and Simplifying One-Shot Architecture Search (2018, unknown): “Copyright 2018 by the author(s). proposed (Brock et al., 2017; Pham et al., 2018; Elsken et al., 2017; Liu et al., 2017a; Cai et al., 2017; Liu et al., 2017b).”
• Deep Learning Architecture Search by Neuro-Cell-Based Evolution with Function-Preserving Mutations (2018, methodology): “64 ? Efficient Architecture Search [3] 10 4.”
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning (2018, background): “Related Works There are several reinforcement-learning-based NAS frameworks (Zoph and Le, 2017a,b; Cai et al., 2017; Baker et al., 2017).”
• Practical Deep Learning Architecture Optimization (2017, methodology): “Network Transformation.”
• ADC: Automated Deep Compression and Acceleration with Reinforcement Learning (2018, methodology): “at−1 and reduced resemble the RNN used by other architecture search approaches [65, 7], since st is affected by st−1.”
• Regularized Evolution for Image Classifier Architecture Search (2018, unknown): “, 2018) or Net2Net transformations (Cai et al., 2017).”
• Inductive Transfer for Neural Architecture Optimization (2019, methodology): “Inspired by this, new methods were developed to reduce the search time by limiting the search space to re-peating cells (Zoph et al., 2017; Zhong et al., 2017), smart reusing (Cai et al., 2017) or sharing of the model weights (Pham et al., 2018; Liu et al., 2018b).”
• Searching for Fast Model Families on Datacenter Accelerators (2021, background): “Neural Architecture Search (NAS) [65, 9, 66, 11], a new paradigm of assembling models automatically, has the potential to bridge the gap.”