Neural Architecture Search with Reinforcement Learning
By Barret Zoph, Quoc V. Le. 2016
5,967 citations recorded by the source corpus.
Neural networks are powerful and flexible models that work well for many difficult learning tasks in image, speech and natural language understanding. Despite their success, neural networks are still hard to design. In this paper, we use a recurrent network to generate the model descriptions of neural networks and train this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. On the CIFAR-10 dataset, our method, starting from scratch, can design a novel network architecture that rivals the best human-invented architecture in terms of test set accuracy. Our CIFAR-10 model achieves a test error rate of 3.65, which is 0.09 percent better and 1.05x faster than the previous state-of-the-art model that used a similar architectural scheme. On the Penn Treebank dataset, our model can compose a novel recurrent cell that outperforms the widely-used LSTM cell, and other state-of-the-art baselines. Our cell achieves a test set perplexity of 62.4 on the Penn Treebank, which is 3.6 perplexity better than the previous state-of-the-art model. The cell can also be transferred to the character language modeling task on PTB and achieves a state-of-the-art perplexity of 1.214.
Subfields: Artificial Intelligence, Model Architectures, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Mohit Bansal
- Yoshua Bengio
- Sergey Levine
- Jianfeng Gao
- Percy Liang
- Dacheng Tao
- Ruslan Salakhutdinov
In Model Architectures
- Quoc V. Le
- Yoshua Bengio
- Alan L. Yuille
- Jiashi Feng
- Chunhua Shen
- Ruslan Salakhutdinov
- Song Han
- Mohit Bansal
In Optimization & Training
- Quoc V. Le
- Frank Hutter
- Yoshua Bengio
- Sergey Levine
- Martin Jaggi
- Pieter Abbeel
- Barret Zoph
- Sebastian U. Stich
Citations in the corpus
- MaskGAN: Better Text Generation via Filling in the ______ (2018, methodology): “Rather, the top performing models are still maximum-likelihood trained models, such as the recent architectures found via neural architecture search in Zoph & Le (2017).”
- Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers (2020, methodology): “More generally, our findings may impact the design of automated methods for solving/optimizing machine learning problems (Feurer et al., 2015; Zoph & Le, 2017; Jaderberg et al., 2017).”
- Parameter-Efficient Transfer Learning for NLP (2019, methodology): “Specifically, we run the single-task Neural AutoML algorithm, similar to Zoph & Le (2017); Wong et al. (2018).”
- AutoGrow: Automatic Layer Growing in Deep Convolutional Networks (2019, neutral): “For example, on CIFAR10 dataset, NAS used 22 , 400 Nvidia K40 GPU hours [39] and neural evolution [28] took 60 , 000 GPU hours [8] to perform the search.”
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “Notably, the effectiveness of model scaling heavily depends on the baseline network; to go even further, we use neural architecture search (Zoph & Le, 2017; Tan et al., 2019) to develop a new baseline network, and scale it up to obtain a family of models, called EfficientNets .”
- A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “For example by learning: architectures Zoph & Le (2017), optimization algorithms Bello et al. (2017), initialization distributions Raghu et al. (2019), or pre-trained data representations.”
- Stand-Alone Self-Attention in Vision Models (2019, background): “The new value transformation is ˜ v ab = ( m p ( a, b, m ) W mV ) x ab where multiple value matrices W mV are combined through a convex combination of factors that are a function of the position of the pixel in its neighborhood p ( a, b, m ) .”
- Attention Augmented Convolutional Networks (2019, methodology): “In particular, the MnasNet was found by neural architecture search [54], using only the highly optimized mobile inverted bottleneck block [36] and the Squeeze-andExcitation operation [17] as the primitives in its search space.”
- Weight Standardization (2019, background): “Recently, Luo et al. [26] use the idea of AutoML [24, 48] to learn how to combine IN, LN, and BN to get a new normalization method.”
- Randaugment: Practical automated data augmentation with a reduced search space (2019, background): “A central premise of learned data augmentation is to construct a small, proxy task that may be reflective of a larger task [52, 53, 4].”
- True Few-Shot Learning with Language Models (2021, background): “Selection criteria can be optimized automatically, e.g. with bayesian optimization [44–46], evolutionary methods [45, 47, 48], reinforcement learning [49], or gradient descent [50–53].”
- Multiview Transformers for Video Recognition (2022, neutral): “Since AlexNet [36] won the Ima-geNet challenge by a large margin, CNNs have been quickly adopted to various vision tasks, their architectures have been refined over many generations [12,27,58,63] and later improved by Neural Architecture Search (NAS) [54,65,85].”
- Dynamic Neural Networks: A Survey (2021, background): “The recent research on neural architecture search (NAS) [9], [10] further speeds up the process of designing more powerful structures.”
- Designing Network Design Spaces (2020, background): “Recently, NAS has received a lot of attention and shown excellent results [34, 18, 29].”
- On Network Design Spaces for Visual Recognition (2019, methodology): “Example model families include standard feedforward networks such as ResNets [8] or the NAS model family from [38, 39].”