Meta-learning for evolutionary parameter optimization of classifiers

By Matthias Reif, Faisal Shafait, Andreas R. Dengel. 2012

173 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Warm-starting can also be used for many aspects of AutoML, yet most research focuses on model selection and tuning (Gomes et al., 2012; De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a, 2015b; Wistuba et al., 2015b; Lindauer & Hutter, 2018).”
Survey on Automated Machine Learning (2019, methodology): “Warm-starting can be used for many aspects of AutoML, yet most research focuses on model selection and tuning (De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a,b; Wistuba et al., 2015b; Lindauer and Hutter, 2018).”
Meta-Learning: A Survey (2018, neutral): “[129] follow a very similar approach, using 15 simple, statistical, and landmarking meta-features.”
Initializing Bayesian Hyperparameter Optimization via Meta-Learning (2015, methodology): “Secondly, meta-learning was used to initialize model-free hyperparameter optimization methods with configurations that previously yielded good performance on similar datasets (Reif, Shafait, and Dengel 2012; Gomes et al. 2012).”
Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters (2014, methodology): “Various types of direct search have been applied to the problem as well, such as genetic algorithms [26], particle swarm optimization [21], and tabu search [10].”
Probabilistic Matrix Factorization for Automated Machine Learning (2017, methodology): “Reif et al. (2012) also use meta-features of the dataset to cold-start the optimization performed by a genetic algorithm.”
Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks (2018, methodology): “This is somewhat similar to hyperparameter optimization across data sets (Reif et al., 2012; Hutter et al., 2015).”
Hyperparameter Importance Across Datasets (2017, methodology): “As a consequence, there has been a lot of recent work and progress on hyperparameter optimization, with methods including random search [2], Bayesian optimization [1, 20, 26, 34, 38], evolutionary optimization [29], meta-learning [6, 16, 30, 32, 40, 41] and banditbased methods [23, 28].”
Learning hyperparameter optimization initializations (2015, methodology): “This is either done by using surrogate models that consider past experiments [9], [13], [10] or by using the information on past experiments to initialize the new search [8], [14], [15].”
Automated Machine Learning: Methods, Systems, Challenges (2019, unknown): “, in combination with evolution [38] or by sequential model-based optimization [37].”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “Most work of this kind focus on hyperparameter tuning, with particle swarm optimization [115], [116], evolutionary algorithm [117], and sequential model-based optimization [5], [118], [119].”
Metalearning: a survey of trends and technologies (2013, methodology): “[51] investigate increasing the accuracy and decreasing runtime of a genetic algorithm for selecting learning parameters for a Support Vector Machine and a Random Forests classifier.”
Auto-sklearn: Efficient and Robust Automated Machine Learning (2019, methodology): “This approach of warm-starting optimization by meta-learning has already been successfully applied before [15, 26, 16], but never to an optimization problem as complex as that of searching the space of instantiations of a full-ﬂedged ML framework.”
SmartML: A Meta Learning-Based Framework for Automated Selection and Hyperparameter Tuning for Machine Learning Algorithms (2019, background): “In our framework, the meta-learning feature is emulating the role of the domain expert in the field of machine learning [4, 11].”
Efficient and Robust Automated Machine Learning (2015, unknown): “First, following successful previous work for low dimensional optimization problems [5, 6, 7], we reason across datasets to identify instantiations of machine learning frameworks that perform well on a new dataset and warmstart Bayesian optimization with them (Section 3.”
Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning (2020, methodology): “A better solution is to warmstart BO with ML pipelines that are expected to work well, as done in the k-nearest dataset (KND) approach of Auto-sklearn 1.0 (Reif et al. (2012); Feurer et al. (2015b,a), see also the related work in Section 6.4.1).”