Combining meta-learning and search techniques to select parameters for support vector machines

By Taciana A. F. Gomes, Ricardo Bastos Cavalcante Prudêncio, Carlos Soares, Andre Luis Debiaso Rossi, André Carlos Ponce de Leon Ferreira de Carvalho. 2012

167 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Warm-starting can also be used for many aspects of AutoML, yet most research focuses on model selection and tuning (Gomes et al., 2012; De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a, 2015b; Wistuba et al., 2015b; Lindauer & Hutter, 2018).”
Survey on Automated Machine Learning (2019, methodology): “any aspects of AutoML, yet most research focuses on model selection and tuning (De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a,b; Wistuba et al., 2015b; Lindauer and Hutter, 2018). (Gomes et al., 2012) Pipeline Structure Meta-learning is also applicable for pipeline structure searc”
Meta-Learning: A Survey (2018, neutral): “, 2017) or evolutionary algorithms (Gomes et al., 2012; Reif et al., 2012).”
Initializing Bayesian Hyperparameter Optimization via Meta-Learning (2015, methodology): “Furthermore, in contrast to existing approaches that initialize direct search algorithms via meta-learning (Gomes et al. 2012; Reif, Shafait, and Dengel 2012), SMBO is a particularly good match for initialization with meta-learning as it can make effective use of all performance data it receives as ”
Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning (2018, background): “[6]), genetic algorithms (GA) (Lorena and de Carvalho [17]), and more recently surrogate-based Bayesian optimization (Denwancker et al.”
Efficient and Robust Automated Machine Learning (2015, unknown): “First, following successful previous work for low dimensional optimization problems [5, 6, 7], we reason across datasets to identify instantiations of machine learning frameworks that perform well on a new dataset and warmstart Bayesian optimization with them (Section 3.”
Efﬁcient and Robust Automated Machine Learning (2015, unknown): “First, following successful previous work for low dimensional optimization problems [5, 6, 7], we reason across datasets to identify instantiations of machine learning frameworks that perform well on a new dataset and warmstart Bayesian optimization with them (Section 3.”
Automated Machine Learning: Methods, Systems, Challenges (2019, unknown): “[22] sample parents from a multi-objective Pareto front using an inverse density.”
Metalearning: a survey of trends and technologies (2013, background): “Rephrasing, the common ground the above definitions share, we propose to define a metalearning system as follows: Definition 1 1.”
Scalable Gaussian process-based transfer surrogates for hyperparameter optimization (2017, methodology): “Since none of these strategies use information from previous experiments, metadata can be added analogously to the SMBO counterpart using in initialization (Gomes et al. 2012;Reif et al. 2012).”
Learning hyperparameter optimization initializations (2015, methodology): “This is either done by using surrogate models that consider past experiments [9], [13], [10] or by using the information on past experiments to initialize the new search [8], [14], [15].”
Auto-sklearn: Efficient and Robust Automated Machine Learning (2019, methodology): “This approach of warm-starting optimization by meta-learning has already been successfully applied before [15, 26, 16], but never to an optimization problem as complex as that of searching the space of instantiations of a full-ﬂedged ML framework.”
Hyperparameter Importance Across Datasets (2017, methodology): “As a consequence, there has been a lot of recent work and progress on hyperparameter optimization, with methods including random search [2], Bayesian optimization [1, 20, 26, 34, 38], evolutionary optimization [29], meta-learning [6, 16, 30, 32, 40, 41] and banditbased methods [23, 28].”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “Most work of this kind focus on hyperparameter tuning, with particle swarm optimization [115], [116], evolutionary algorithm [117], and sequential model-based optimization [5], [118], [119].”
Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters (2014, methodology): “Secondly, metalearning was used to initialize hyperparameter optimization methods with hyperparameter configurations that previously yielded good performance on similar datasets [26, 21, 10].”
Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization (2015, background): “The third optional component is some initialization technique which usually starts which a hyperparameter con guration that has proven to be good on many data sets [9,11].”