BOHB: Robust and Efficient Hyperparameter Optimization at Scale

By Stefan Falkner, Aaron Klein, Frank Hutter. 2018

1,347 citations recorded by the source corpus.

Modern deep learning methods are very sensitive to many hyperparameters, and, due to the long training times of state-of-the-art models, vanilla Bayesian hyperparameter optimization is typically computationally infeasible. On the other hand, bandit-based configuration evaluation approaches based on random search lack guidance and do not converge to the best configurations as quickly. Here, we propose to combine the benefits of both Bayesian optimization and bandit-based methods, in order to achieve the best of both worlds: strong anytime performance and fast convergence to optimal configurations. We propose a new practical state-of-the-art hyperparameter optimization method, which consistently outperforms both Bayesian optimization and Hyperband on a wide range of problem types, including high-dimensional toy functions, support vector machines, feed-forward neural networks, Bayesian neural networks, deep reinforcement learning, and convolutional neural networks. Our method is robust and versatile, while at the same time being conceptually simple and easy to implement.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Management Science and Operations Research, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Quoc V. Le
  2. Yoshua Bengio
  3. Jianfeng Gao
  4. Frank Hutter
  5. Mohit Bansal
  6. Sergey Levine
  7. Percy Liang
  8. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Yi Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Ming-Hsuan Yang
  7. Cordelia Schmid
  8. Jitendra Malik

In Management Science and Operations Research

  1. Frank Hutter
  2. Aaron Klein
  3. Stefan Falkner
  4. Ameet Talwalkar
  5. Matthias Feurer
  6. Liam Li
  7. Afshin Rostamizadeh
  8. Marc-André Zöller

In Neural Architecture Search & AutoML

  1. Frank Hutter
  2. Colin White
  3. Yash Savani
  4. Aaron Klein
  5. Xuanyi Dong
  6. Marius Thomas Lindauer
  7. Stefan Falkner
  8. Arber Zela

Citations in the corpus