Model-based methods for continuous and discrete global optimization

By Thomas Bartz-Beielstein, Martin Zaefferer. 2017

161 citations recorded by the source corpus.

Graphical abstractDisplay Omitted HighlightsUp-to-date survey and comprehensive taxonomy of surrogate model based optimization algorithms.Covers continuous and discrete/combinatorial search spaces.Presents six strategies for dealing with discrete data structures.New strategy for model selection and combination in surrogate model-based optimization.Outlook on important challenges (model selection, dimensionality, benchmarks, definiteness) and research directions. The use of surrogate models is a standard method for dealing with complex real-world optimization problems. The first surrogate models were applied to continuous optimization problems. In recent years, surrogate models gained importance for discrete optimization problems. This article takes this development into consideration. The first part presents a survey of model-based methods, focusing on continuous optimization. It introduces a taxonomy, which is useful as a guideline for selecting adequate model-based optimization tools. The second part examines discrete optimization problems. Here, six strategies for dealing with discrete data structures are introduced. A new approach for combining surrogate information via stacking is proposed in the third part. The implementation of this approach will be available in the open source R package SPOT2. The article concludes with a discussion of recent developments and challenges in continuous and discrete application domains.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Control and Systems Engineering

Citations in the corpus

Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor (2020, methodology): “Since the CNN architecture is presented as discrete variables, random forest [52], which is suited for discrete regression tasks and also relies on only limited labeled data [53], is adopted as the fitness predictor in the proposed algorithm.”
Linear Combination of Distance Measures for Surrogate Models in Genetic Programming (2018, unknown): “Frequently, Surrogate Model-Based Optimization (SMBO) is used to deal with expensive evaluations [4].”
Distance-based Kernels for Surrogate Model-based Neuroevolution (2018, background): “While surrogate models are frequently employed in continuous optimization, more complex discrete search spaces are less often investigated [3].”
A Random Forest-Assisted Evolutionary Algorithm for Data-Driven Constrained Multiobjective Combinatorial Optimization of Trauma Systems (2020, methodology): “Compared with RBF models, RF models have not been widely used as surrogates, although it has recently been suggested that the tree structure in RF models is well suited for approximating functions with discrete decision variables [34].”
A Survey of Automatic Parameter Tuning Methods for Metaheuristics (2020, methodology): “Bartz-Beielstein and Zaefferer [88] have demonstrated the superiority of SPO over SKO, and thus SKO appears rarely in parameter tuning domain.”
Data-Driven Evolutionary Optimization: An Overview and Case Studies (2019, background): “However, many real-world online data-driven problems are constrained [91]–[96] and/or of mixed-integer decision variables [97]–[103].”