An evaluation of adaptive surrogate modeling based optimization with two benchmark problems

By Chen Wang, Qingyun Duan, Wei Gong, Aizhong Ye, Zhenhua Di, Chiyuan Miao. 2014

252 citations recorded by the source corpus.

Surrogate modeling uses cheap ''surrogates'' to represent the response surface of simulation models. It involves several steps, including initial sampling, regression and adaptive sampling. This study evaluates an adaptive surrogate modeling based optimization (ASMO) method on two benchmark problems: the Hartman function and calibration of the SAC-SMA hydrologic model. Our results show that: 1) Gaussian Processes are the best surrogate model construction method. A minimum Interpolation Surface method is the best adaptive sampling method. Low discrepancy Quasi Monte Carlo methods are the most suitable initial sampling designs. Some 15-20 times the dimension of the problem may be the proper initial sample size; 2) The ASMO method is much more efficient than the widely used Shuffled Complex Evolution global optimization method. However, ASMO can provide only approximate optimal solutions, whose precision is limited by surrogate modeling methods and problem-specific features; and 3) The identifiability of model parameters is correlated with parameter sensitivity.

Subfields: Computational Theory and Mathematics, Ocean Engineering, Statistics and Probability, Optimization & Training, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

1. Xin He
2. Olivier Bousquet
3. Sylvain Gelly
4. Wei Gong
5. Karol Kurach
6. Kaiyong Zhao
7. Chen Wang
8. Qingyun Duan

In Ocean Engineering

1. Sylvain Gelly
2. Wei Gong
3. Qingyun Duan

In Statistics and Probability

1. Yongjiu Dai
2. Xin He
3. Olivier Bousquet
4. Sylvain Gelly
5. Wei Gong
6. Karol Kurach
7. Kaiyong Zhao
8. Chen Wang

In Optimization & Training

1. Xin He
2. Olivier Bousquet
3. Sylvain Gelly
4. Karol Kurach
5. Kaiyong Zhao
6. Damien Vincent
7. Xiaowen Chu

In Evaluation & Benchmarks

1. Sylvain Gelly
2. Xin He
3. Olivier Bousquet
4. Wei Gong
5. Karol Kurach
6. Kaiyong Zhao
7. Chen Wang
8. Qingyun Duan

Citations in the corpus

• AutoML: A Survey of the State-of-the-Art (2019, methodology): “A surrogate-based method [133, 134, 135] is another powerful tool that approximates to the black-box function.”
• An Intercomparison of Sampling Methods for Uncertainty Quantification of Environmental Dynamic Models (2015, methodology): “Parameters of the SAC-SMA Model and their Feasible Ranges and Assigned Values (Wang et al., 2014)”
• Critical Hyper-Parameters: No Random, No Cry (2017, methodology): “Results in [24] already advocated LDS for the initialization of GP, pointing out that this outperforms Latin Hypercube Sampling; in the present section, we confirm those results in the case of deep learning and show that we also outperform typical pessimistic fantasizing as used in batch entropy…”