Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters

By Matthias Feurer, Jost Tobias Springenberg, Frank Hutter. 2014

79 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2016, methodology): “1 117 datasets We use the framework introduced by Feurer et al. (2015a), which explores a structured hyperparameter search space with a total of 110 hyperparameters comprised of 15 classifiers, 14 feature preprocessing methods, and 4 data preprocessing methods. Similar to Feurer et al. (2015a), we i”
Meta-Learning: A Survey (2018, neutral): “[48] offer a simpler, more scalable method that warm-starts Bayesian optimization by sorting all prior tasks tj similar to [59], but including 46 simple, statistical, and landmarking meta-features, as well as H(C).”
Initializing Bayesian Hyperparameter Optimization via Meta-Learning (2015, positive): “This paper is an improved and extended version of a previous workshop submission (Feurer, Springenberg, and Hutter 2014)”
λOpt: Learn to Regularize Recommender Models in Finer Levels (2019, background): “Previous work [11, 21, 28, 30, 31] have dedicated to lower down the nontrivial search cost, along with developing some enhanced toolboxes [4, 7, 23].”
Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization (2015, background): “The third optional component is some initialization technique which usually starts which a hyperparameter con guration that has proven to be good on many data sets [9,11].”
Towards Automatic Composition of Multicomponent Predictive Systems (2016, methodology): “The search space size can be reduced by applying constraints like limiting the number of components, restricting the list of methods using meta-learning [21], prior knowledge [22] or surrogates [23].”
Scalable Gaussian process-based transfer surrogates for hyperparameter optimization (2017, background): “Feurer et al. (2014) propose the same idea for SMBO which was later extended (Feurer et al. 2015; Wistuba et al. 2015).”
Learning hyperparameter optimization initializations (2015, methodology): “This is either done by using surrogate models that consider past experiments [9], [13], [10] or by using the information on past experiments to initialize the new search [8], [14], [15].”
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (2020, methodology): “Two key factors driving auto-sklearn’s success include its use of meta-learning to warm start the hyperparameter search (Feurer et al., 2014), as well as combining many models via the ensemble selection strategy of Caruana et al. (2004).”
Automated Machine Learning in Practice: State of the Art and Recent Results (2019, background): “tion (SMAC) [35] or the Tree-structured Parzen Estimator [36].”