Optimization as Estimation with Gaussian Processes in Bandit Settings

By Zi Wang, Bolei Zhou, Stefanie Sabrina Jegelka. 2015

80 citations recorded by the source corpus.

Recently, there has been rising interest in Bayesian optimization -- the optimization of an unknown function with assumptions usually expressed by a Gaussian Process (GP) prior. We study an optimization strategy that directly uses an estimate of the argmax of the function. This strategy offers both practical and theoretical advantages: no tradeoff parameter needs to be selected, and, moreover, we establish close connections to the popular GP-UCB and GP-PI strategies. Our approach can be understood as automatically and adaptively trading off exploration and exploitation in GP-UCB and GP-PI. We illustrate the effects of this adaptive tuning via bounds on the regret as well as an extensive empirical evaluation on robotics and vision tasks, demonstrating the robustness of this strategy for a range of performance criteria.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2016, methodology): “cesses have also been studied in the bandit setting using condence bound acquisition functions (GP-UCB), with associated sublinear regret bounds (Srinivas et al., 2010; Grunew alder et al., 2010). Wang et al. (2016) improved upon GP-UCB by removing the need to tune a parameter that controls explorat”
On Kernelized Multi-armed Bandits (2017, background): “Other Bayesian approaches to function optimization are GP-EI (Močkus, 1975), GP-PI (Kushner, 1964), GP-EST (Wang et al., 2016) and GP-UCB, including the contextual (Krause & Ong, 2011), high-dimensional (Djolonga et al.”
Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design (2019, background): “A key advantage of GPs is that inference is very efficient, which makes them one of the most popular theoretical tools for Bayesian optimization (Rasmussen & Williams, 2006; Srinivas et al., 2010; Wang et al., 2016).”
Max-value Entropy Search for Efficient Bayesian Optimization (2017, methodology): “Such upper bounds still lead to optimization strategies with vanishing regret, whereas lower bounds may not (Wang et al., 2016).”
Depth Creates No Bad Local Minima (2017, methodology): “…decoupled training problems, with each having a small number of weight parameters, we can effectively train a model via non-convex optimization as theoretically shown in Bayesian optimization and global optimization literatures (Kawaguchi et al., 2015; Wang et al., 2016; Kawaguchi et al., 2016).”