A Taxonomy of Global Optimization Methods Based on Response Surfaces

By Donald R. Jones. 2001

2,409 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Management Science and Operations Research, Statistics and Probability, Evaluation & Benchmarks, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Management Science and Operations Research

In Statistics and Probability

In Evaluation & Benchmarks

In Mathematical Optimization

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “Typically, Gaussian processes have been used to construct the distribution over functions used in Bayesian optimization, due to their ﬂexibility, well-calibrated uncertainty, and analytic properties (Jones, 2001; Os-borne et al., 2009).”
A Comprehensive Survey of Neural Architecture Search (2020, background): “gate model f̂ (performance predictor) evaluates all candidate cells α (the structure description of the neural architecture) [37] and selects promising candidate cells, and then evaluates the performance of the neural architecture composed of the candidate cells on the validation set and gets the re”
A Survey on Neural Architecture Search (2019, methodology): “As the name implies, surrogate model-based optimizers use a surrogate model f̂ to approximate the response function f (Equation (8)) (Jones, 2001).”
Fast AutoAugment (2019, methodology): “Precisely, at the line 6 in Algorithm 1, we employ the following Expected Improvement (EI) criterion [18] for acquisition function to explore candidate policies B efficiently: EI(T ) = E [ min(L(θ|T (DA))− L†, 0) ] = ∫ min(L − L†, 0)Pθ,DA(L|T )dL (4) Here the expectation in (4) is taken over the den”
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets (2016, methodology): “To maximize the acquisition function we used the blackbox optimizer DIRECT (Jones, 2001) and CMAES (Hansen, 2006).”
An evaluation of adaptive surrogate modeling based optimization with two benchmark problems (2014, methodology): “However, it may take a long time to do so (Jones, 2001).”
Freeze-Thaw Bayesian Optimization (2014, background): “A popular and effective acquisition function is the expected improvement (EI) criterion [15, 16], where φ and Φ , respectively, are the probability density function and cumulative distribution function of the standard normal distribution, x best is the input corresponding to the minimum output…”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, methodology): “Bayesian optimization techniques are some of the most efficient approaches in terms of the number of function evaluations required (see, e.g. [Močkus, 1994, Jones et al., 1998, Streltsov and Vakili, 1999, Jones, 2001, Sasena, 2002]).”
Automated Machine Learning on Big Data using Stochastic Algorithm Tuning (2014, neutral): “Bayesian optimisation [3, 12] applies probabilistic modelling to global optimisation.”
Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction (2015, background): “Bayesian optimization is efficient in terms of the number of function evaluation [24], and is particularly effective when f is computationally expensive.”
Bayesian Optimization with Unknown Constraints (2014, background): “When the predictive distribution under the model is Gaussian, EI has a closed-form expression (Jones, 2001):”
A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems (2014, methodology): “Among them, GP modeling, response surface methods, artificial neural networks, support vector machines, and radial basis functions exhibit good performances and are widely used [7], [8], [10], [12]–[16].”
Theory and Use of the EM Algorithm (2011, background): “For more on state-of-the-art global optimization, see for example [1, 23, 24, 28, 38, 46].”
A review of recent advances in global optimization (2009, background): “Jones (2001) provided a comparative overview of the various response surface approaches, with regard to their use in global optimization.”