MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning

By Chi-Hung Hsu, Shu-Huan Chang, Da-Cheng Juan, Jia-Yu Pan, Yu-Ting Chen, Wei Wei, Shih-Chieh Chang. 2018

149 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Neural Architecture Search & AutoML

Citations in the corpus

The Efficiency Misnomer (2021, methodology): “To this end, these studies mostly use the parameter size (Pham et al., 2018), FLOPs (Hsu et al., 2018), memory access cost (MAC) (Ma et al., 2018), or real latency (Tan et al., 2019; Wu et al., 2019).”
MnasNet: Platform-Aware Neural Architecture Search for Mobile (2018, methodology): “Closely related to our work is MONAS [12], DPP-Net [3], RNAS [34] and Pareto-NASH [4] which attempt to optimize multiple objectives, such as model size and accuracy, while searching for CNNs, but their search process optimizes on small tasks like CIFAR.”
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours (2019, methodology): “More recent methods incorporate hardware terms ( e.g. , runtime, power) into cell-based NAS formulations [10,12], but cell-based implementations are not hardware friendly [21].”
Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search (2019, background): “1) Cell Diversity: Our experiments show that a good cell diversity also helps to achieve better results for superresolution, same for classification tasks [20].”
Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks (2018, methodology): “These types of approaches are often called multiobjective or platform-aware neural architecture search (Hsu et al., 2018; Zhou et al., 2018; Tan et al., 2018; Dong et al., 2018).”
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware (2018, background): “Therefore, it is a multi-objective NAS task (Hsu et al., 2018; Dong et al., 2018; Elsken et al., 2018a; He et al., 2018; Tan et al., 2018), where one of the objectives is non-differentiable (i.”
Graph HyperNetworks for Neural Architecture Search (2018, methodology): “While earlier NAS methods focused on standard image classiﬁcation and language modeling, recent literature has extended NAS to search for architectures that are computationally efﬁcient (Tan et al., 2018; Dong et al., 2018; Hsu et al., 2018; Elsken et al., 2018; Zhou et al., 2018).”
NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search (2018, methodology): “State-of-the-art reinforcement learning (RL) Baker et al. (2017); Zhong et al. (2017); Zoph & Le (2016); Zoph et al. (2017); Hsu et al. (2018a); Cai et al. (2018); Pham et al. (2018) and evolutionary algorithm (EA) Miikkulainen et al. (2017); Real et al. (2017; 2018); Xie & Yuille (2017); Kim et…”
Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search (2019, methodology): “Since the reward is obtained after model m is evaluated, which ends an episode, we adopt REINFORCEMENT to update this controller (Sutton & Barto, 2018).”
Differentiable Architecture Search with Ensemble Gumbel-Softmax (2019, methodology): “Additionally, a series of well-performance methods have also been explored, including the progressive search (Liu et al., 2018a) and multi-objective optimization (Tan et al., 2018; Hsu et al., 2018).”
Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS (2020, methodology): “Recently, Neural Architecture Search has been used intensively to find architectures that have better tradeoff between accuracy and latency [38, 41, 8, 5, 37], FLOPS [39], power consumption [16], and memory usage [10].”
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better (2021, methodology): “Other works such as MONAS [64] use reinforcement learning to incorporate power consumption into the reward function along with hard constraints on the number of MAC operations in the model, and discover Pareto-frontiers under the given constraints.”
DATA: Differentiable ArchiTecture Approximation With Distribution Guided Sampling (2020, methodology): “Additionally, a series of well-performing methods have also been explored, including progressive search [34] and multi-objective optimization [21], [39].”
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization (2019, methodology): “More recent methods incorporate hardware terms ( e.g. , runtime, power) into cell-based NAS formulations [29], [31], but cell-based implementations are not hardware friendly [16].”
Macro Neural Architecture Search Revisited (2018, methodology): “This greedy approach is most similar to multi-objective architecture search (Elsken et al., 2018; Hsu et al., 2018) that focuses on models on the pareto-frontier of the performance scatter plot.”
Deep Reinforcement Learning for Multiobjective Optimization (2019, background): “However, they mainly focus on how to apply RL to control a robot with multiple goals, such as controlling a mountain car or controlling a submarine searching for treasures, as investigated in [23].”
InstaNAS: Instance-aware Neural Architecture Search (2018, background): “To name a few, [5, 8, 24, 30] aims at finding neural architectures that satisfy architecture-dependent metrics by achieving Pareto optimality over more than one objective.”
NSGA-Net: neural architecture search using multi-objective genetic algorithm (2018, unknown): “Hsu et al. [15] extends the NASNet approach to multi-objective domain to optimize multiple linear combinations of accuracy and energy consumption criteria for different scalarization parameters.”
CARS: Continuous Evolution for Efficient Neural Architecture Search (2019, unknown): “MONAS [20] targets at accuracy and energy.”