Practical bayesian optimization

By Daniel J. Lizotte. 2008

320 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, background): “For an in-depth review, see Lizotte (2008), Brochu et al. (2010) and Os-borne et al. (2009).”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “In this setting, we consider a sequential search algorithm which, at iteration n , selects a location x n +1 at which to query f and observe y n +1 .”
Freeze-Thaw Bayesian Optimization (2014, methodology): “rantees [6,7,8], more rigorous methodologies [9,10] and methods tailored to speciﬁc applications such as hyperparameter optimization [11,4,3] and sensor set selection [12]. Bayesian optimization (see [13,14] for an in-depth explanation) proceeds by ﬁtting a probabilistic model to the data and then u”
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010, methodology): “This is sometimes called “one-step” [Močkus, 1994] “average-case” [Streltsov and Vakili, 1999] or “practical” [Lizotte, 2008] optimization.”
Entropy Search for Information-Efficient Global Optimization (2011, background): “But it is useful to pause at this point to contrast this deﬁnition with other concepts of optimization.”
Gaussian Processes for Data-Efficient Learning in Robotics and Control (2015, neutral): “[10] and Bayesian optimization [9], [33], [41], [51] to avoid getting stuck in local minima.”
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions (2014, methodology): “We perform another series of experiments in which we optimize well-known synthetic benchmark functions including a mixture of cosines [2] and Branin-Hoo (both functions defined in [0, 1](2)) as well as the Hartmann-6 (defined in [0, 1](6)) [15].”
A Tutorial on Bayesian Optimization (2018, methodology): “Lizotte (2008) proposed using gradient information in this way in Bayesian optimization, together with the EI acquisition function, showing an improvement over BFGS (Liu and Nocedal, 1989).”
Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks (2015, methodology): “We experimented with various model architectures and hyper-parameter settings using Bayesian optimization, implemented in the package Spearmint (available from https://github.com/HIPS/Spearmint) (Snoek et al. 2012).”
A Bayesian interactive optimization approach to procedural animation design (2010, methodology): “Previous methods have dealt with this by initializing the data set with a large number of random samples from a Latin hypercube [SWN03], or having a system expert select values for the hyperparameters or hyperposteriors [Liz08], neither of which is suitable to our application.”
Adaptive MCMC with Bayesian Optimization (2012, methodology): “We refer readers to Brochu et al. (2009) and Lizotte (2008) for in-depth reviews of Bayesian optimization.”
Bayesian Optimization in a Billion Dimensions via Random Embeddings (2013, background): “More precisely, the function whose optimum we seek is f ( x 1: D ) = b ( x i , x j ), where b is the Branin function (see Lizotte, 2008, for its exact formula), and where dimensions i and j are selected once using a random permutation of 1 , . . . , D .”
Programming by Feedback (2014, methodology): “…approach relies on learning surrogate models, predicting the performance yielded by a portfolio algorithm or an algo-rithm conﬁguration depending on the context, and using sequential model-based optimization ( Lizotte, 2008) to determine the best conﬁguration for the particular problem instance.”
Learning dynamic arm motions for postural recovery (2011, methodology): “To achieve cost scale invariance, the maximum likelihood mean was computed analytically after each trial and used in the log likelihood computation [8].”
An experimental methodology for response surface optimization methods (2011, background): “…surface methods has been steady over the past 40 years (Kushner 1964; Perttunen et al. 1989; Stuckman 1989; Mockus 1989; Sacks et al. 1989; Elder 1992; Cox and John 1995; Schonlau 1997; Jones and Schonlau 1998; Gutmann 2001; Sasena 2002; Santner et al. 2003; Boyle 2006; Regis 2007; Lizotte 2008).”
UvA-DARE (Digital Academic Repository) Bayesian Optimization in High Dimensions via Random Embeddings (2013, unknown): “More precisely, the function whose optimum we seek is f ( x 1: D ) = g ( x i , x j ) , where g is the Branin function (for its exact formula, see [ Lizotte, 2008 ] ) and where i and j are selected once using a random permutation.”
Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations (2012, background): “We refer the reader to (Močkus, 1982; Schonlau et al., 1998; Gramacy et al., 2004; Brochu et al., 2007; Lizotte, 2008; Martinez–Cantin et al., 2009; Garnett et al., 2010) for practical examples.”