SMASH: One-Shot Model Architecture Search through HyperNetworks

By Andrew Brock, Theodore Lim, James M. Ritchie, Nick Weston. 2017

808 citations recorded by the source corpus.

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at this https URL

Subfields: Artificial Intelligence, Information Systems and Management, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Dacheng Tao
  5. Jianfeng Gao
  6. Sergey Levine
  7. Percy Liang
  8. Trevor Darrell

In Information Systems and Management

  1. Wenpeng Hu
  2. Xiang Ren
  3. Tal Ben-Nun
  4. Song Han
  5. Doina Precup
  6. Graham W. Taylor
  7. Frank Hutter
  8. Qi Tian

In Neural Architecture Search & AutoML

  1. Frank Hutter
  2. Xuanyi Dong
  3. Quoc V. Le
  4. Martin Wistuba
  5. Arber Zela
  6. Xiaojun Chang
  7. Song Han
  8. Thomas Elsken

Citations in the corpus