Beyond Manual Tuning of Hyperparameters

By Frank Hutter, Jörg Lücke, Lars Schmidt-Thieme. 2015

187 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bernt Schiele
2. Yaoyao Liu
3. Qianru Sun
4. Iryna Gurevych
5. Nils Reimers
6. Massimiliano Pontil
7. Jason H. Moore
8. Luca Franceschi

In Computational Theory and Mathematics

1. Nicolás Navarro-Guerrero
2. Jason H. Moore
3. Paolo Frasconi
4. Massimiliano Pontil
5. Iryna Gurevych
6. Stefan Wermter
7. Ryan J. Urbanowicz
8. Tobias Hinz

Citations in the corpus

• Forward and Reverse Gradient-Based Hyperparameter Optimization (2017, background): “The increasing complexity of machine learning algorithms has driven a large amount of research in the area of hyperparameter optimization (HO) — see, e.g., (Hutter et al., 2015) for a review.”
• Automating Biomedical Data Science Through Tree-Based Pipeline Optimization (2016, background): “Historically, automated machine learning pipeline optimization has focused on optimizing specific elements of the pipeline [6].”
• Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks (2017, background): “Ad hoc manual tuning is still a commonly and often surprisingly effective approach for hyperparameter tuning (Hutter et al., 2015).”
• An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, methodology): “n weight mare used to configure the m-th base-learner. It is well known that fine-grained hyperparameters, e.g. layerwise learning rates, are more efficient, but exponentially expensive to set by hand [21, 5]. Our LCC does not have this problem and can learn fine-grained hyperparameters without additio”
• Bilevel Programming for Hyperparameter Optimization and Meta-Learning (2018, methodology): “Classic approaches to HO (see e.g. Hutter et al., 2015, for a survey) have been only able to manage a relatively small number of hyperparameters, from a few dozens using random search (Bergstra and Bengio, 2012) to a few hundreds using Bayesian or model-based approaches (Bergstra et al., 2013;…”
• Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks (2018, methodology): “This is somewhat similar to hyperparameter optimization across data sets (Reif et al., 2012; Hutter et al., 2015).”
• Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science (2016, background): “Historically, machine learning automation research has primarily focused on optimizing subsets of the pipeline [10].”