Warmstarting of Model-based Algorithm Configuration

By Marius Thomas Lindauer, Frank Hutter. 2017

70 citations recorded by the source corpus.

The performance of many hard combinatorial problem solvers depends strongly on their parameter settings, and since manual parameter tuning is both tedious and suboptimal the AI community has recently developed several algorithm configuration (AC) methods to automatically address this problem. While all existing AC methods start the configuration process of an algorithm A from scratch for each new type of benchmark instances, here we propose to exploit information about A's performance on previous benchmarks in order to warmstart its configuration on new types of benchmarks. We introduce two complementary ways in which we can exploit this information to warmstart AC methods based on a predictive model. Experiments for optimizing a flexible modern SAT solver on twelve different instance sets show that our methods often yield substantial speedups over existing AC methods (up to 165-fold) and can also find substantially better configurations given the same compute budget.

Subfields: Computational Theory and Mathematics, Computer Networks and Communications, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Computer Networks and Communications

In Industrial and Manufacturing Engineering

Citations in the corpus

Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Warm-starting can be used for many aspects of AutoML, yet most research focuses on model selection and tuning (De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a,b; Wistuba et al., 2015b; Lindauer and Hutter, 2018).”
Survey on Automated Machine Learning (2019, methodology): “Warm-starting can be used for many aspects of AutoML, yet most research focuses on model selection and tuning (De Miranda et al., 2012; Reif et al., 2012; Feurer et al., 2015a,b; Wistuba et al., 2015b; Lindauer and Hutter, 2018).”
Transferable AutoML by Model Sharing Over Grouped Datasets (2019, methodology): “Furthermore, to compare the collaborative way for AutoML, we compare Tr-AutoML with a warmstart method: [24] by using the three AutoML methods as its user-specified default initialization.”
Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon (2018, methodology): “The automatic conﬁguration optimization algo-rithm used by Lindauer and Hutter (2018) requires training an empirical cost model mapping the Cartesian product of parameter conﬁgurations and problem instances to expected algorithmic performance.”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “Most work of this kind focus on hyperparameter tuning, with particle swarm optimization [115], [116], evolutionary algorithm [117], and sequential model-based optimization [5], [118], [119].”
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges (2021, methodology): “In both cases, it is possible to warm-start the HPO process itself — for example, by choosing the initial design as HPCs that have performed well in the past or by choosing HPCs that are known to perform well in general (Lindauer & Hutter, 2018).”