NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search

By Zhichao Lu, Ian Whalen, Vishnu Naresh Boddeti, Yashesh D. Dhebar, Kalyanmoy Deb, Erik D. Goodman, Wolfgang Banzhaf. 2018

88 citations recorded by the source corpus.

This paper introduces NSGA-Net, an evolutionary approach for neural architecture search (NAS). NSGA-Net is designed with three goals in mind: (1) a NAS procedure for multiple, possibly conflicting, objectives, (2) efficient exploration and exploitation of the space of potential neural network architectures, and (3) output of a diverse set of network architectures spanning a trade-off frontier of the objectives in a single run. NSGA-Net is a population-based search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures and finally an exploitation step that applies the entire history of evaluated neural architectures in the form of a Bayesian Network prior. Experimental results suggest that combining the objectives of minimizing both an error metric and computational complexity, as measured by FLOPS, allows NSGA-Net to find competitive neural architectures near the Pareto front of both objectives on two different tasks, object classification and object alignment. NSGA-Net obtains networks that achieve 3.72% (at 4.5 million FLOP) error on CIFAR-10 classification and 8.64% (at 26.6 million FLOP) error on the CMU-Car alignment task. Code available at: this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Neural Architecture Search & AutoML

Citations in the corpus

A Comprehensive Survey of Neural Architecture Search (2020, background): “ickly applied in the fields of object detection [65, 75, 111, 118], semantic segmentation [63, 64, 120], adversarial learning [53], architectural scaling [114, 122, 124], multi-objective optimization [39, 115, 125], platform-aware [28, 34, 103, 117], data augmentation [121, 123] and so on. In additi”
GreedyNAS: Towards Fast One-Shot NAS With Greedy Supernet (2020, methodology): “For example, ScarletNAS [3] employs a multi-objective searcher [22] to consider classification error, FLOPs and model size for better paths.”
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search (2019, background): “Abundant efforts have been dedicated to various methods that guide the search within carefully designed search space [30, 16, 24, 13, 25].”
Automated Machine Learning: State-of-The-Art and Open Challenges (2019, background): “Some recent approaches consider using the multi-objective evolutionary architecture search to optimize training time, complexity and performance [76, 36] of the network.”
A Survey on Neural Architecture Search (2019, methodology): “In their experiments Lu et al. (2018) report results on both NASNet search space and the genotype representation by Xie and Yuille (2017) (see Section 2 for details) and therefore are not limited to chainstructured networks.”
Evaluating the Search Phase of Neural Architecture Search (2019, background): “, 2018), evolutionary algorithms (Xie & Yuille, 2017; Real et al., 2017; Miikkulainen et al., 2019; Liu et al., 2018b; Lu et al., 2018) and performance predictors (Liu et al.”
Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search (2019, methodology): “Similar to [8], [7], we also apply NSGA-II [9] to solve the multi-objective problem.”
Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search (2019, methodology): “Moreover, this approach constructs a one-toone mapping between the cell code and neural architectures, possessing an inherent advantage over NSGA-Net, whose space design involves a many-to-one mapping that further repetition removal steps must be taken to avoid meaningless training labors (Lu et al.”
Evolutionary neural AutoML for deep learning (2019, background): “ution [46], for searching network topologies. Others rely on genetic programming [47] or hierarchical evolution [31]. There is some very recent work on multiobjective evolutionary architecture search [18, 33], where the goal is to optimize both the performance and training time/complexity of the net”
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search (2019, background): “[19] Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, and Wolfgang Banzhaf.”
SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search (2019, methodology): “volution works. Searching with Multi-Objective NAS Pipeline Upon a ready-to-use scalable supernet, we take advantage of a multi-objective approach to serve as our search pipeline as (Deb et al. 2002; Lu et al. 2019; Chu et al. 2019). Here we consider three objectives: classiﬁcation error rate, multi”
ScarletNAS: Bridging the Gap Between Scalability and Fairness in Neural Architecture Search (2019, methodology): “Searching with Multi-Objective NAS Pipeline Upon a ready-to-use scalable supernet, we take advantage of a multi-objective approach to serve as our search pipeline as (Deb et al. 2002; Lu et al. 2019; Chu et al. 2019).”