Designing Neural Network Architectures using Reinforcement Learning
By Bowen Baker, Otkrist Gupta, Nikhil Naik, Ramesh Raskar. 2016
1,553 citations recorded by the source corpus.
At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $\epsilon$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.
Subfields: Artificial Intelligence, Reinforcement Learning, Neural Architecture Search & AutoML
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Mohit Bansal
- Yoshua Bengio
- Sergey Levine
- Dacheng Tao
- Jianfeng Gao
- Trevor Darrell
- Alan L. Yuille
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Martin Wistuba
- Trevor Darrell
- Ruslan Salakhutdinov
- Igor Mordatch
- Abhishek Gupta
In Neural Architecture Search & AutoML
Citations in the corpus
- Neural Architecture Optimization (2018, methodology): “The latest algorithms for automatic architecture design usually fall into two categories: reinforcement learning (RL) [47, 48, 36, 3] based methods and evolutionary algorithm (EA) based methods [42, 34, 39, 27, 38].”
- Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution (2018, methodology): “Neural Architecture Search Recently, NAS was framed as a reinforcement learning (RL) problem, where the reward of the RL agent is based on the validation performance of the trained architecture (Baker et al., 2017a; Zoph & Le, 2017; Zhong et al., 2018; Pham et al., 2018).”
- Neural Optimizer Search with Reinforcement Learning (2017, methodology): “Our approach is reminiscent of recent work in automated model discovery with Reinforcement Learning (Baker et al., 2016), especially Neural Architecture Search (Zoph & Le, 2017), in which a recurrent network is used to generate the configuration string of neural architectures instead.”
- Paraphrasing Complex Network: Network Compression via Factor Transfer (2018, methodology): “Also, there are methods of designing a network using a reinforcement learning algorithm such as MetaQNN [2] and Neural Architecture Search [33].”
- AutoHAS: Differentiable Hyper-parameter and Architecture Search (2020, background): “Since the seminal works [1, 55] show promising improvements over manually designed architectures, more efforts have been devoted to NAS.”
- A Comprehensive Survey of Neural Architecture Search (2020, methodology): “The work of NAS-RL [11] and MetaQNN [12] is considered a pioneering work of NAS.”
- Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor (2020, background): “As a result, the potential promising architecture may not be found through EDLs within a limited computational budget due to the time-consuming fitness evaluation.”
- BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, background): “Since the popular NASNet paper [54], there has been a flurry of research on neural architecture search [27, 34, 29, 17, 6, 15, 36, 3, 1, 28].”
- Understanding and Robustifying Differentiable Architecture Search (2019, methodology): “Prior work mostly employs either reinforcement learning techniques [Baker et al., 2017a, Zoph and Le, 2017, Zhong et al., 2018, Zoph et al., 2018] or evolutionary algorithms [Stanley and Miikkulainen, 2002, Liu et al., 2018, Miikkulainen et al., 2017, Real et al., 2017, 2019] to optimize the…”
- Auto-GNN: Neural architecture search of graph neural networks (2019, background): “Most of NAS frameworks are built up based on one of the two basic algorithms: RL [12, 13, 15, 38, 39] and EA [14, 40–43].”
- AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, neutral): “For the optimization al-gorithm, popular options include reinforcement learning [4, 71, 69, 72], evolutionary algorithm [60], Bayesian optimization [24], random search [7], and gradient-based optimization methods [38, 1].”
- PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search (2019, background): “Another line of heuristics turns to reinforcement learning (RL) (Zoph & Le, 2017; Baker et al., 2017; Zoph et al., 2018; Zhong et al., 2018; Liu et al., 2018a), which trained a meta-controller to guide the search process.”
- Evolving Robust Neural Architectures to Defend from Adversarial Attacks (2019, methodology): “Most of the algorithms for AAS are either based on reinforcement learning [44],[45],[27], [3] or evolutionary computation [41], [22], [29],[17], [28].”
- Continual and Multi-Task Architecture Search (2019, methodology): “Some recent attempts have made architecture search more computationally feasible (Negrinho and Gordon, 2017; Baker et al., 2017) via tree-structured search space or Q-learning with an (cid:15) - greedy exploration, and further improvements via a weight-sharing strategy called Efficient Neural…”
- Automated Machine Learning: State-of-The-Art and Open Challenges (2019, methodology): “[4] introduced a metamodeling algorithm called MetaQNN based on reinforcement learning to automatically generate the architecture of convolutional neural network for a new task.”
- Searching for a Robust Neural Architecture in Four GPU Hours (2019, methodology): “Recently, neural architecture search (NAS) has increasingly attracted the interest of researchers [1, 6, 17, 22, 25, 31, 46].”