Sequential parameter optimization

By Thomas Bartz-Beielstein, Christian Lasarczyk, Mike Preuss. 2005

347 citations recorded by the source corpus.

Sequential parameter optimization is a heuristic that combines classical and modern statistical techniques to improve the performance of search algorithms. To demonstrate its flexibility, three scenarios are discussed: (1) no experience how to choose the parameter setting of an algorithm is available, (2) a comparison with other algorithms is needed, and (3) an optimization algorithm has to be applied effectively and efficiently to a complex real-world optimization problem. Although sequential parameter optimization relies on enhanced statistical techniques such as design and analysis of computer experiments, it can be performed algorithmically and requires basically the specification of the relevant algorithm's parameters

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “The so-called Bayesian Optimization (BO) (a.k.a. Eﬃcient Global Optimization) [23, 33] algorithm has been applied extensively for automated algorithm conﬁguration tasks [2, 21, 22].”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, background): “body of work devoted to extending the algorithm, particularly in adding constraints to the optimization problem [6], [23], [131], and in modeling noisy functions [14], [75], [76].”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, background): “It is also possible to resample potential incumbents to get more reliable estimates of the values in a noisy environment [Bartz-Beielstein et al., 2005, Huang et al., 2006, Hutter et al., 2009], a process sometimes called intensification.”
Online Batch Selection for Faster Training of Neural Networks (2015, methodology): “However, the question of optimal hyperparameter settings is of great interest and we plan to investigate it further with CMA-ES (Hansen & Ostermeier, 2001) and standard Bayesian optimization methods (Bartz-Beielstein et al., 2005; Snoek et al., 2012; Bergstra et al., 2011; Hutter et al., 2011).”
Algorithm Runtime Prediction: Methods and Evaluation (Extended Abstract) (2015, methodology): “First, they can model the performance of a parameterized algorithm dependent on the settings of its parameters; in a sequential model-based optimization process, one alternates between learning an EPM and using it to identify promising settings to evaluate next [Jones et al., 1998; Bartz-Beielstein ”
Exploration and exploitation in evolutionary algorithms: A survey (2013, neutral): “…approach (evolution of evolution) [Grefenstette 1986], racing algorithm [Birattari et al. 2002], sequential parameter optimization [Bartz-Beielstein et al. 2005], meta-estimation of distribution algorithm [Nannen and Eiben 2006], hyper-heuristics [Ochoa et al. 2008],…”
Efficient global optimization for combinatorial problems (2014, methodology): “Categories and Subject Descriptors G.1.6 [Mathematics of Computing]: Optimization—Global Optimization; G.2.1 [Discrete Mathematics]: Combinatorics—Combinatorial Algorithms General Terms Algorithms, Experimentation Keywords Genetic Algorithm, Surrogate Model-Based Optimization, Efficient Global Optim”
The irace package: Iterated racing for automatic algorithm configuration (2016, methodology): “Among the ﬁrst surrogate-based conﬁguration methods is sequential parameter optimization (SPOT) [12] .”
A Survey of Automatic Parameter Tuning Methods for Metaheuristics (2020, methodology): “The commonly known model-based tuning approaches, including the sequential parameter optimization (SPO) [92] and the more sophisticated sequential model-based algorithm conﬁguration (SMAC) [47].”
Comparing parameter tuning methods for evolutionary algorithms (2009, background): “Mercer and Sampson [13] were the first to introduce a meta-EA, but due to the large computational costs, their research was very limited.”
Meta-learning for evolutionary parameter optimization of classifiers (2012, methodology): “Bartz-Beielstein et al. (2005) presented an approach for optimizing the parameters of search heuristics which can also be used to optimize different parameters of genetic algorithms.”
Surrogate-Assisted Learning of Neural Networks (2017, methodology): “No algorithm is able to deliver their best performance for every problem without adapting their control parameters; By parameter tuning [2, 6], we can exploit beneﬁcial parameter settings, but it is very time-demanding.”
Automated Algorithm Configuration and Parameter Tuning (2012, methodology): “[8] follows a fundamentally different strategy to deal with noisy response measurements:3 Rather than fitting a Gaussian process model directly to a set of sampled”
Automated configuration of algorithms for solving hard computational problems (2009, methodology): “…problem; see Section 3.1.1 SPO Sequential Parameter Optimization, a model-based framework for stochastic continuous blackbox optimization first described by Bartz-Beielstein et al. (2005) SKO Sequential Kriging Optimization, a model-based approach for stochastic continuous blackbox optimization…”
Algorithm runtime prediction: Methods & evaluation (2012, methodology): “Such RSMs are at the core of sequential model-based optimization methods for blackbox functions [65], which have been adapted to applications in automated parameter tuning and algorithm conﬁguration (see, e.g. , [7,6,58,59,55]).”
Multi-Objective Parameter Selection for Classifiers (2012, background): “Sequential parameter optimization (Bartz-Beielstein et al. 2005; Bartz-Beielstein 2006) is another parameter tuning framework that tries to cope with the stochastically disturbed results of search heuristics by repeated evaluations.”
Sequential Model-Based Optimization for General Algorithm Configuration (2011, methodology): “Despite considerable recent advances [13, 14, 15], all published work on SMBO still has three key limitations that prevent its use for general algorithm configuration tasks: (1) it only supports numerical parameters; (2) it only optimizes target algorithm performance for”