SGAS: Sequential Greedy Architecture Search

By Guohao Li, Guocheng Qian, Itzel C. Delgadillo, Matthias Müller, Ali K. Thabet, Bernard Ghanem. 2019

210 citations recorded by the source corpus.

Architecture design has become a crucial component of successful deep learning. Recent progress in automatic neural architecture search (NAS) shows a lot of promise. However, discovered architectures often fail to generalize in the final evaluation. Architectures with a higher validation accuracy during the search phase may perform worse in the evaluation. Aiming to alleviate this common issue, we introduce sequential greedy architecture search (SGAS), an efficient method for neural architecture search. By dividing the search procedure into sub-problems, SGAS chooses and prunes candidate operations in a greedy fashion. We apply SGAS to search architectures for Convolutional Neural Networks (CNN) and Graph Convolutional Networks (GCN). Extensive experiments show that SGAS is able to find state-of-the-art architectures for tasks such as image classification, point cloud classification and node classification in protein-protein interaction graphs with minimal computational cost.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Materials Chemistry

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Materials Chemistry

Citations in the corpus

MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation (2020, background): “For example, task-agnostic NAS is proposed for the multi-task learning framework [6]; releasing the constraints of handdesigned heuristics [29] or alleviating the gap between the search accuracy and the evaluation accuracy are also promising directions [3, 14].”
A Comprehensive Survey of Neural Architecture Search (2020, methodology): “Although we have done many reviews, there are still many improvements based on DARTS [113, 151].”
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search (2019, unknown): “08y 97.35 RL ENAS [27] 4.6 626y 97.11 RL MdeNAS[40] 3.6 599y 97.45 MDL DARTS(second order)?[23] 3.3 528y 97.24 0:09 GD SNAS? [37] 2.8 422y 97.15 0:02 GD GDAS [12] 3.37 519y 97.07 GD SGAS (Cri.2 avg.) [19] 3.9 y0.22y 640 39 97.33 0:21 GD P-DARTS [6] 3.4 532y 97.5 GD PC-DARTS [38] 3.6 558y 97.43 GD RD”
DARTS-: Robustly Stepping out of Performance Collapse Without Indicators (2020, background): “SGAS (Li et al., 2020), instead, circumvents the problem with a greedy strategy where the unfair advantage can be prevented from taking effect.”
Parameter Prediction for Unseen Deep Architectures (2021, methodology): “± 0.09 ) accuracy, which is competitive with leading NAS approaches, e.g. [19, 24, 32, 67–69].”
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020, background): “Lately, researchers have applied neural architecture search to 3D medical image segmentation [78, 22, 72, 2, 65, 74] and 3D shape classification [34, 26].”
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (2021, methodology): “…both weight-sharing based methods (Liu et al., 2018b; Dong & Yang, 2019; Liu et al., 2019; Yu et al., 2020a; Li et al., 2020a; Yang et al., 2020a) and single-path sampling-based methods (Pham et al., 2018; Guo et al., 2019; Real et al., 2019; Tan et al., 2020; Li et al., 2020c; Yang et al., 2020b).”