Experimental Research in Evolutionary Computation - The New Experimentalism

By Thomas Bartz-Beielstein. Natural Computing Series, 2010

296 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/3c871e59b63f625846185092843f1358f673bfdc

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The irace package: Iterated racing for automatic algorithm configuration (2016, methodology): “The methods used in this context include experimental design techniques [2,29] , racing approaches [20] , and algorithmic methods for parameter conﬁguration, such as heuristic search techniques [3,10,41,73,81] , and statistical modeling approaches [11,43] .”
A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization (2009, methodology): “• The experimental design consists of a set of techniques which comprise methodologies for adjusting the parameters of the algorithms depending on the settings used and results obtained (Bartz-Beielstein 2006; Kramer 2007).”
Sequential Model-Based Optimization for General Algorithm Configuration (2011, background): “…π′′ ∈ Π} 5 π ← instance sampled uniformly at random from Π ′ 6 s← seed, drawn uniformly at random 7 R← ExecuteRun(R, θinc, π, s) 8 N ← 1 9 while true do 10 Smissing ← 〈instance, seed〉 pairs for which θinc was run before, but not θnew 11 Storun ←random subset of Smissing of size min(N, |Smissing”
Automated configuration of algorithms for solving hard computational problems (2009, methodology): “For example, they can be used to determine how much algorithm response depends on the setting of single parameters, and to which degree parameters interact (Jones et al., 1998; Santner et al., 2003; Bartz-Beielstein, 2006).”
A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms (2011, background): “Single-problem and multi-problem analyses can usually be found contrasting the results of computational intelligence experiments, both in isolation [17] and simultaneously [18].”
Parameter tuning for configuring and analyzing evolutionary algorithms (2011, background): “For instance, [4] uses the names qualitative, and quantitative parameters, [5] distinguishes symbolic and numeric parameters, [6] calls them categorical and numerical, while [7] refers to structural and behavioral parameters.”
Identifying Key Algorithm Parameters and Instance Features Using Forward Selection (2013, methodology): “Notable exceptions in the literature include experimental design based on linear models [11,12], an entropy-based measure [2], and visualization methods for interactive parameter exploration, such as contour plots [13].”
Automated Configuration of Mixed Integer Programming Solvers (2010, background): “Much existing work is restricted to scenarios having only relatively small numbers of numerical (often continuous) parameters, both in the model-based [7, 13, 17] and model-free [6, 1] literatures.”
A Field Guide to Genetic Programming (2008, unknown): “4) can significantly speed up the process, tools from the design of experiments literature (Bartz-Beielstein, 2006) can also be used to reduce the number of different runs that are necessary to explore the space in a statistically sound manner.”
Algorithm runtime prediction: Methods & evaluation (2012, methodology): “Other approaches for quantifying parameter importance include an entropy-based measure [85], and visualization methods for interactive parameter exploration [6].”
ParamILS: An Automatic Algorithm Configuration Framework (2014, methodology): “Sequential parameter optimization (SPO) (Bartz-Beielstein, 2006) is a model-based parameter optimization approach based on the Design and Analysis of Computer Experiments (DACE; see, e.g., Santner, Williams & Notz, 2003), a prominent approach in statistics for blackbox function optimization.”
Multi-Objective Parameter Selection for Classifiers (2012, background): “Sequential parameter optimization (Bartz-Beielstein et al. 2005; Bartz-Beielstein 2006) is another parameter tuning framework that tries to cope with the stochastically disturbed results of search heuristics by repeated evaluations.”