A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

By Eric Brochu, Vlad M. Cora, Nando de Freitas. 2010

2,681 citations recorded by the source corpus.

We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utility-based selection of the next observation to make on the objective function, which must take into account both exploration (sampling from areas of high uncertainty) and exploitation (sampling areas likely to offer improvement over the current best observation). We also present two detailed extensions of Bayesian optimization, with experiments---active user modelling with preferences, and hierarchical reinforcement learning---and a discussion of the pros and cons of Bayesian optimization based on our experiences.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Management Science and Operations Research, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Sergey Levine
  3. Mohit Bansal
  4. Quoc V. Le
  5. Jianfeng Gao
  6. Ruslan Salakhutdinov
  7. Percy Liang
  8. Frank Hutter

In Electrical and Electronic Engineering

  1. Sergey Levine
  2. Frank Hutter
  3. Zoubin Ghahramani
  4. Aaron Klein
  5. Katharina Eggensperger
  6. Ziyun Wang
  7. Jost Tobias Springenberg
  8. Andreas Krause

In Management Science and Operations Research

  1. Frank Hutter
  2. Aaron Klein
  3. Nando de Freitas
  4. Peter I. Frazier
  5. Kirthevasan Kandasamy
  6. Stefan Falkner
  7. Jeff Schneider
  8. Ziyun Wang

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Jan Peters
  5. Shimon Whiteson
  6. Yuxi Li
  7. Shixiang Shane Gu
  8. Marc Peter Deisenroth

In Optimization & Training

  1. Frank Hutter
  2. Aaron Klein
  3. Stefan Falkner
  4. Quoc V. Le
  5. Marc-André Zöller
  6. Marco F. Huber
  7. Ameet Talwalkar
  8. Yoshua Bengio

Citations in the corpus