CARS: Continuous Evolution for Efficient Neural Architecture Search

By Zhaohui Yang, Yunhe Wang, Xinghao Chen, Boxin Shi, Chao Xu, Chunjing Xu, Qi Tian. 2019

269 citations recorded by the source corpus.

Searching techniques in most of existing neural architecture search (NAS) algorithms are mainly dominated by differentiable methods for the efficiency reason. In contrast, we develop an efficient continuous evolutionary approach for searching neural networks. Architectures in the population that share parameters within one SuperNet in the latest generation will be tuned over the training dataset with a few epochs. The searching in the next evolution generation will directly inherit both the SuperNet and the population, which accelerates the optimal network generation. The non-dominated sorting strategy is further applied to preserve only results on the Pareto front for accurately updating the SuperNet. Several neural networks with different model sizes and performances will be produced after the continuous search with only 0.4 GPU days. As a result, our framework provides a series of networks with the number of parameters ranging from 3.7M to 5.1M under mobile settings. These networks surpass those produced by the state-of-the-art methods on the benchmark ImageNet dataset.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Qi Tian
3. Luc Van Gool
4. Yunhe Wang
5. Zhangyang Wang
6. Xiaogang Wang
7. Alan L. Yuille
8. Yi Yang

Citations in the corpus

• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation (2020, methodology): “The first method relies on evolutionary algorithms [23, 5, 30].”
• VEGA: Towards an End-to-End Configurable AutoML Pipeline (2020, methodology): “We implement CARS (Yang et al., 2020) with VEGA and evaluate the models on Cifar-10(Krizhevsky et al., 2009).”
• Adversarially Robust Neural Architectures (2020, unknown): “proposed an efficient continuous evolutionary approach on the supernet where all the architectures share the parameters, which boosted the searching efficiency [53].”
• One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting (2020, methodology): “Then, a heuristic search method finds promising architectures over a discrete search space based on the trained supernet [20], [30], [46], [59].”
• Distilling Object Detectors via Decoupled Features (2021, background): “Designing a detector coupled with lightweight backbone network [55, 41, 45, 32, 61, 64] is also a trend for faster detection speed.”
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (2021, methodology): “…both weight-sharing based methods (Liu et al., 2018b; Dong & Yang, 2019; Liu et al., 2019; Yu et al., 2020a; Li et al., 2020a; Yang et al., 2020a) and single-path sampling-based methods (Pham et al., 2018; Guo et al., 2019; Real et al., 2019; Tan et al., 2020; Li et al., 2020c; Yang et al., 2020b).”
• A Survey on Evolutionary Neural Architecture Search (2020, methodology): “But from the perspective of building the architecture, CARS can also be classified into the cell-based category.”
• Parameter Prediction for Unseen Deep Architectures (2021, methodology): “± 0.09 ) accuracy, which is competitive with leading NAS approaches, e.g. [19, 24, 32, 67–69].”
• Vision GNN: An Image is Worth Graph of Nodes (2022, methodology): “The representative works include ResNet [17], MobileNet [21] and NAS [75, 70].”
• GhostNet: More Features From Cheap Operations (2019, methodology): “This kind of methods can also provide new search unit for automatic search methods [62, 55, 5].”