Differentiable Architecture Search with Ensemble Gumbel-Softmax

By Jianlong Chang, Xinbang Zhang, Yiwen Guo, Gaofeng Meng, Shiming Xiang, Chunhong Pan. 2019

20 citations recorded by the source corpus.

For network architecture search (NAS), it is crucial but challenging to simultaneously guarantee both effectiveness and efficiency. Towards achieving this goal, we develop a differentiable NAS solution, where the search space includes arbitrary feed-forward network consisting of the predefined number of connections. Benefiting from a proposed ensemble Gumbel-Softmax estimator, our method optimizes both the architecture of a deep network and its parameters in the same round of backward propagation, yielding an end-to-end mechanism of searching network architectures. Extensive experiments on a variety of popular datasets strongly evidence that our method is capable of discovering high-performance architectures, while guaranteeing the requisite efficiency during searching.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Gaofeng Meng
2. Yukang Chen
3. Tong Yang
4. X. Zhang
5. Xinyu Xiao
6. Quoc V. Le
7. Yi Yang
8. Yoshua Bengio

In Computer Vision and Pattern Recognition

1. Yukang Chen
2. Gaofeng Meng
3. Tong Yang
4. Xinyu Xiao
5. X. Zhang
6. Xiaodan Liang
7. Andrea Vedaldi
8. Dacheng Tao

Citations in the corpus

• DetNAS: Backbone Search for Object Detection (2019, background): “Many following works, including SNAS [28], Proxyless [3] and FBNet [27] and others [4], also belong to one-shot NAS to some extent.”