HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking

By Shen Yan, Biyi Fang, Faen Zhang, Yu Zheng, Xiao Zeng, Hui Xu, Mi Zhang. 2019

22 citations recorded by the source corpus.

The use of automatic methods, often referred to as Neural Architecture Search (NAS), in designing neural network architectures has recently drawn considerable attention. In this work, we present an efficient NAS approach, named HM-NAS, that generalizes existing weight sharing based NAS approaches. Existing weight sharing based NAS approaches still adopt hand designed heuristics to generate architecture candidates. As a consequence, the space of architecture candidates is constrained in a subset of all possible architectures, making the architecture search results sub-optimal. HM-NAS addresses this limitation via two innovations. First, HM-NAS incorporates a multi-level architecture encoding scheme to enable searching for more flexible network architectures. Second, it discards the hand designed heuristics and incorporates a hierarchical masking scheme that automatically learns and determines the optimal architecture. Compared to state-of-the-art weight sharing based approaches, HM-NAS is able to achieve better architecture search performance and competitive model evaluation accuracy. Without the constraint imposed by the hand designed heuristics, our searched networks contain more flexible and meaningful architectures that existing weight sharing based NAS approaches are not able to discover.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chaoyang He
2. Haishan Ye
3. Li Shen
4. Tong Zhang
5. Peter Richtárik
6. Murali Annavaram
7. Jakub Konecný
8. Martin Jaggi

In Computer Vision and Pattern Recognition

1. Chaoyang He
2. Tong Zhang
3. Li Shen
4. Haishan Ye
5. Andrew Zisserman
6. Luc Van Gool
7. Dacheng Tao
8. Andrea Vedaldi

Citations in the corpus

• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation (2020, background): “For example, task-agnostic NAS is proposed for the multi-task learning framework [6]; releasing the constraints of handdesigned heuristics [29] or alleviating the gap between the search accuracy and the evaluation accuracy are also promising directions [3, 14].”