Efficient Global Optimization of Expensive Black-Box Functions

By Donald R. Jones, Matthias Schonlau, William J. Welch. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Mathematical Optimization

Citations in the corpus

Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, background): “where σn(x) > 0, φ(·) and Φ(·) are the standard normal probability density function and cumulative standard normal distribution function; and μn(x) is the best observed value thus far (Jones et al., 1998).”
Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “The so-called Bayesian Optimization (BO) (a.k.a. Eﬃcient Global Optimization) [23, 33] algorithm has been applied extensively for automated algorithm conﬁguration tasks [2, 21, 22].”
Automated Machine Learning: State-of-The-Art and Open Challenges (2019, methodology): “Bayesian Optimization is one of the state-of-the-art blackbox optimization techniques which is tailored for expensive objective functions [69, 134, 88, 55].”
An evaluation of adaptive surrogate modeling based optimization with two benchmark problems (2014, background): “For example, S obester et al. (2005) suggested that about 35% of the samples should be spent in the initial stage, while Jones et al. (1998) recommended that the initial sample size should be about ten times the dimensionality of the problem.”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “The typical estimation of the hyperparameters by maximizing the marginal likelihood [126], [82] can easily fall into traps, as shown in [32].”
Grammar Variational Autoencoder (2017, methodology): “After training the SGP, we then perform 5 iterations of batch Bayesian optimization using the expected improvement (EI) heuristic (Jones et al., 1998).”
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules (2016, neutral): “After this, we performed 10 iterations of Bayesian optimization using the expected improvement (EI) heuristic [49].”
Learning to Learn for Global Optimization of Black Box Functions (2016, background): “…2012, Russo and Van Roy, 2014, Shahriari et al., 2014, Hern ´ andez-Lobato et al., 2015], probability of improvement [Kushner, 1964, Jones et al., 1998], expected improvement [Moˇckus et al., 1978, Jones et al., 1998, Jones, 2001, Brochu et al., 2009, Snoek et al., 2012, Wang and de Freitas,…”
Meta-Learning: A Survey (2018, neutral): “expected improvement (Jones et al., 1998) on Pi,new and the predicted improvements on all prior Pi,j.”
Initializing Bayesian Hyperparameter Optimization via Meta-Learning (2015, methodology): “A classic approach is to initialize SMBO with a space-filling design (Jones, Schonlau, and Welch 1998).”
Multi-fidelity Bayesian Optimisation with Continuous Approximations (2017, methodology): “We compare BOCA to the following four baselines: (i) GP-UCB , (ii) the GP-EI criterion in BO (Jones et al., 1998), (iii) MF-GP-UCB (Kandasamy et al., 2016a) and (iv) MF-SKO , the multi-ﬁdelity sequential kriging optimisation method from Huang et al. (2006).”
APRIL: Active Preference-learning based Reinforcement Learning (2012, neutral): “They further note that returning the optimal solution, e.g. using the Expected Global Improvement criterion [17] with a branch-and-bound method, raises technical issues on high-dimensional search spaces.”
ParamILS: An Automatic Algorithm Configuration Framework (2014, background): “The metric underlying the choice of promising parameter configurations is the expected improvement criterion (Jones et al., 1998).”
Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit Optimization (2012, methodology): “We compare it with four alternatives: (1) The strictly sequential GP-UCB algorithm (B = 1); (2) NRB-UCB, an approach that simply picks the maximizer of the GP-UCB score B times; (3) NTB-UCB, an approach that picks the top B scores according to the GP-UCB criterion; (4) A state of the art algorithm f”