Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization

By Martin Wistuba, Nicolas Schilling, Lars Schmidt-Thieme. 2015

78 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Marco F. Huber
2. Marc-André Zöller
3. Frank Hutter
4. Martin Wistuba
5. Lars Schmidt-Thieme
6. Cédric Archambeau
7. Qiang Yang
8. Nicolas Schilling

In Computational Theory and Mathematics

1. Marco F. Huber
2. Marc-André Zöller
3. Lars Schmidt-Thieme
4. Martin Wistuba
5. Frank Hutter
6. Nicolas Schilling
7. Jan N. van Rijn
8. Huibin Shen

Citations in the corpus

• Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Warm-starting can also be used for many aspects of AutoML, yet most research focuses on model selection and tuning (Gomes et al., 2012; De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a, 2015b; Wistuba et al., 2015b; Lindauer & Hutter, 2018).”
• Survey on Automated Machine Learning (2019, methodology): “Meta-learning can be used to filter empirically bad performing candidate configurations (Alia and Smith-Miles, 2006; Wistuba et al., 2015a).”
• Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “Existing works of this line make effort to evaluate the importance of configurations [120], [121], or identify promising regions in the search space [122].”
• Learning hyperparameter optimization initializations (2015, methodology): “This is either done using an initialization for SMBO [8], a specific surrogate model that is learning across experiments [9], [10], [11] or search space pruning [12].”
• Hyperparameter Importance Across Datasets (2017, methodology): “nfiguration for the dataset at hand. These techniques have been successfully used to recommend good hyperparameter settings [30, 35], to warm-start optimization procedures [13] or prune search spaces [44]. However, it is hard to select an adequate set of meta-features. Moreover, obtaining good meta-”
• Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning (2019, methodology): “The work most closely related to ours is [49], where the search space is pruned during BO, removing unpromising regions based on information from related BO problems.”