Heteroscedastic Treed Bayesian Optimisation
By Yannis Assael, Ziyun Wang, Nando de Freitas. 2014
48 citations recorded by the source corpus.
Optimising black-box functions is important in many disciplines, such as tuning machine learning models, robotics, finance and mining exploration. Bayesian optimisation is a state-of-the-art technique for the global optimisation of black-box functions which are expensive to evaluate. At the core of this approach is a Gaussian process prior that captures our belief about the distribution over functions. However, in many cases a single Gaussian process is not flexible enough to capture non-stationarity in the objective function. Consequently, heteroscedasticity negatively affects performance of traditional Bayesian methods. In this paper, we propose a novel prior model with hierarchical parameter learning that tackles the problem of non-stationarity in Bayesian optimisation. Our results demonstrate substantial improvements in a wide range of applications, including automatic machine learning and mining exploration.
Subfields: Artificial Intelligence, Management Science and Operations Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Frank Hutter
- Nando de Freitas
- Sergey Levine
- Ziyun Wang
- Philipp Hennig
- Ryan P. Adams
- Kevin Swersky
- Aaron Klein
In Management Science and Operations Research
- Frank Hutter
- Ziyun Wang
- Ryan P. Adams
- Bobak Shahriari
- Kevin Swersky
- Nando de Freitas
- Javier I. González
- Philipp Hennig
Citations in the corpus
- Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “to handling heteroscedastic noise was proposed in [5] using a partitioning approach.”
- Scalable Global Optimization via Local Bayesian Optimization (2019, background): “Previous work on non-stationary kernels does not make this assumption, but these approaches are too computationally expensive to be applicable in our large-scale setting [43, 46, 3].”
- Batch Bayesian Optimization via Local Penalization (2015, result): “Although this is a typical hypothesis for most BO methods, recent works have pointed out that some real problems do not satisfy this condition [Assael et al., 2014].”