NAS-Bench-101: Towards Reproducible Neural Architecture Search

By Chris Ying, Aaron Klein, Esteban Real, Eric Christiansen, Kevin P. Murphy, Frank Hutter. 2019

821 citations recorded by the source corpus.

Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public architecture dataset for NAS research. To build NAS-Bench-101, we carefully constructed a compact, yet expressive, search space, exploiting graph isomorphisms to identify 423k unique convolutional architectures. We trained and evaluated all of these architectures multiple times on CIFAR-10 and compiled the results into a large dataset of over 5 million trained models. This allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. We demonstrate its utility by analyzing the dataset as a whole and by benchmarking a range of architecture optimization algorithms.

Subfields: Artificial Intelligence, Information Systems, Materials Chemistry, Evaluation & Benchmarks, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Mohit Bansal
  3. Yoshua Bengio
  4. Jianfeng Gao
  5. Dacheng Tao
  6. Sergey Levine
  7. Percy Liang
  8. Luke S. Zettlemoyer

In Information Systems

  1. Colin White
  2. Yash Savani
  3. Frank Hutter
  4. Xuanyi Dong
  5. Sam Nolen
  6. Linnan Wang
  7. Tie-Yan Liu
  8. Tao Qin

In Materials Chemistry

  1. Yash Savani
  2. Colin White
  3. Sam Nolen
  4. Frank Hutter
  5. Willie Neiswanger
  6. Xuanyi Dong
  7. Marius Thomas Lindauer
  8. Xia Hu

In Evaluation & Benchmarks

  1. Frank Hutter
  2. Colin White
  3. Yash Savani
  4. Sam Nolen
  5. Marius Thomas Lindauer
  6. Xuanyi Dong
  7. Yi Yang
  8. Quoc V. Le

In Neural Architecture Search & AutoML

  1. Colin White
  2. Yash Savani
  3. Frank Hutter
  4. Sam Nolen
  5. Willie Neiswanger
  6. Marius Thomas Lindauer
  7. Xuanyi Dong
  8. Quoc V. Le

Citations in the corpus