Portfolio Allocation for Bayesian Optimization

By Matthew D. Hoffman, Eric Brochu, Nando de Freitas. 2010

317 citations recorded by the source corpus.

Bayesian optimization with Gaussian processes has become an increasingly popular tool in the machine learning community. It is efficient and can be used when very little is known about the objective function, making it popular in expensive black-box optimization scenarios. It uses Bayesian methods to sample the objective efficiently using an acquisition function which incorporates the model's estimate of the objective and the uncertainty at any given point. However, there are several different parameterized acquisition functions in the literature, and it is often unclear which one to use. Instead of using a single acquisition function, we adopt a portfolio of acquisition functions governed by an online multi-armed bandit strategy. We propose several portfolio strategies, the best of which we call GP-Hedge, and show that this method outperforms the best individual acquisition function. We also provide a theoretical bound on the algorithm's performance.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Management Science and Operations Research

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, background): “For an in-depth review, see Lizotte (2008), Brochu et al. (2010) and Os-borne et al. (2009).”
Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, methodology): “It could be designed based on the hedge algorithm, using which the acquisition function was chosen with past performance (Hoffman et al., 2011).”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, background): “[73] proposed the use of a portfolio containing multiple acquisition strategies.”
Freeze-Thaw Bayesian Optimization (2014, background): “Recent work has developed compelling theoretical convergence guarantees [6, 7, 8], more rigorous methodologies [9, 10] and methods tailored to speciﬁc applications such as hyperparameter optimization [11, 4, 3] and sensor set selection [12].”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, methodology): “For convenience, we assume here that the prior mean is the zero func- tion m(x) = 0; alternative priors for the mean can be found in, for example [Martinez–Cantin et al., 2009, Brochu et al., 2010a].”
Input Warping for Bayesian Optimization of Non-Stationary Functions (2014, background): “Recent work in machine learning has revisited the idea of Bayesian optimization [e.g., Osborne et al., 2009, Brochu et al., 2010, Srinivas et al., 2010, Hutter et al., 2011, Bergstra et al., 2011, Bull, 2011, Snoek et al., 2012, Hennig and Schuler, 2012] in large part due to advances in the ability…”
Learning Where to Attend with Deep Architectures for Image Tracking (2011, methodology): “There are many selection methods available in the literature that offer different trade-offs between these two criteria (Hoffman, Brochu, & de Freitas, 2011 ).”
Metacontrol for Adaptive Imagination-Based Optimization (2017, background): “B depicts an easy, 1-planet scene, while C depicts a very difﬁcult 5-planet scene. several basic selection criteria Hoffman et al. (2011); Shahriari et al. (2014).”
Active Reward Learning (2014, methodology): “[16] that we used to optimize the model of the reward function.”
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions (2014, background): “By conditioning on D n , the posterior for θ is also multivariate Gaussian, Let φ ( i ) and θ ( i ) be a random set of features and the corresponding posterior weights sampled both according to the generative process given above.”
High Dimensional Bayesian Optimisation and Bandits via Additive Models (2015, methodology): “Following, Brochu et al. (2010) we use DiRect to maximise ϕt, ϕ̃t.”
AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning (2019, methodology): “While one could select the points that maximize the mean function, this does not always lead to the best outcome (Hoffman et al., 2011).”
Adaptive MCMC with Bayesian Optimization (2012, methodology): “A comparison among these options as well as portfolio strategies to combine them appeared recently in (Hoffman et al., 2011).”
Bayesian Optimization in a Billion Dimensions via Random Embeddings (2013, methodology): “It is also possible to use dynamic portfolios of acquisition functions to improve the eﬃciency of the method (Hoﬀman et al., 2011).”
On Kernelized Multi-armed Bandits (2017, background): “We compare GP-UCB, IGP-UCB and GP-TS with GP-EI and GP-PI 11 .”
BayesOpt: a Bayesian optimization library for nonlinear optimization, experimental design and bandits (2014, methodology): “BayesOpt also implements metacriteria algorithms, like the bandit algorithm GP-Hedge by Hoffman et al. (2011) that can be used to automatically select the most suitable criteria during the optimization.”
A new acquisition function for Bayesian optimization based on the moment-generating function (2017, methodology): “In [25], [26], it is proposed to use portfolio strategies to select an acquisition function in each step of the Bayesian optimization.”