UvA-DARE (Digital Academic Repository) Bayesian Optimization in High Dimensions via Random Embeddings

By Ziyun Wang, Masrour Zoghi, Frank Hutter, David Matheson, Nando de Freitas. 2013

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, background): “Recent work has sought to improve the performance of the GP approach through accommodating higher dimensional problems (Wang et al., 2013; Djolonga et al., 2013), input non-stationarities (Snoek et al., 2014) and initialization through meta-learning (Feurer et al., 2015).”
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2016, methodology): “the effective dimension (Wang et al., 2013) or the number of additive components (Kandasamy et al.”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, neutral): “We can denote these data as a set of tuples D n = { ( a i , y i ) } ni =1 , where a i indicates which of the K drugs was administered and y i is 1 if the patient was cured and 0 otherwise.”
Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models (2015, background): “In recent work, it was assumed that the objective varies along a low dimensional subspace only (Chen et al., 2012; Wang et al., 2013; Djolonga et al., 2013), or that it takes the form of an additive model (Kandasamy et al.”
Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters (2013, background): “As a result, hyperparameter optimization has become an active research area within BO, with characteristics such as low effective dimensionality [1] [5] [6] and problem variants, such as optimization across different data sets [7] being explored.”
Derivative-Free Optimization via Classification (2016, methodology): “RACOS is compared with simultaneous optimistic optimization (SOO) algorithm (Munos 2011; 2014), random embedding Bayesian optimization (REMBO) algorithm (Wang et al. 2013), and covariance matrix adaptation evolution strategy (CMA-ES) algorithm (Hansen, Müller, and Koumoutsakos 2003), where the imple”
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining (2020, methodology): “Early formulations of LSO were motivated by scaling Gaussian processes to high dimensional problems with simple linear manifolds, using either random projections [60] or a learned transformation matrix [12].”
Constrained Bayesian Optimization for Automatic Chemical Design (2017, unknown): “LP) [126, 129] and Statistical Machine Translation (SMT) [4], Robotics and Reinforcement Learning [67, 78, 77, 17, 12], Combinatorial Optimization problems such as the tuning of mixed integer solvers [46, 127, 125] and nearest neighbour algorithms [75] and interactive interfaces for preference learn”
Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics (2016, unknown): “However, this can be scaled up by discovering a low-dimensional subspace of A for Bayesian optimization (Djolonga et al. 2013; Wang et al. 2013) or encoding additional structure in the kernel (Duvenaud et al.”
High-Dimensional Gaussian Process Bandits (2013, background): “The idea of random projections to BO has been recently introduced [19].”
Bayesian Optimization for Synthetic Gene Design (2015, methodology): “This will require, however, the development of scalable Bayesian optimization methods able to work well in high dimensions in the line of some recent works [17, 5, 2, 7].”
Bayesian Optimization for Adaptive Experimental Design: A Review (2020, methodology): “BO has been influential in computer science for hyperparameter tuning [38]–[42], combinatorial optimisation [43], [44], and reinforcement learning [21].”
BayesOpt: a Bayesian optimization library for nonlinear optimization, experimental design and bandits (2014, methodology): “We also provide a method for optimization in high-dimensional spaces (Wang et al., 2013).”
Active Domain Randomization (2019, background): “While seemingly just an instantiation of the traditional BO problem, the nonstationarity of the objective (the environment utility) requires us to redefine the notion of an acquisition function while simultaneously dealing with BO’s deficiencies with higher-dimensional inputs (Wang et al., 2013).”
An Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems (2020, methodology): “…search space of LOPs, they cannot be easily solved by general evolutionary algorithms, hence some novel techniques have been adopted to tackle speciﬁc types of LOPs, such as variable interaction analysis [32]–[34], linkage learning [35], and random embedding-based Bayesian optimization [36], [37].”
An Efficient Approach for Assessing Hyperparameter Importance (2014, background): “Recent work on Bayesian optimization has targeted the case where most hyperparameters are truly unimportant (Chen et al., 2012; Wang et al., 2013), and several applications have yielded evidence that some hyperparameters indeed tend to be much more important than others (Bergstra and Bengio, 2012;…”
AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning (2018, background): “(Wang et al., 2013) identifies a low-dimensional effective subspace for the black-box function via random embedding.”
High Dimensional Bayesian Optimisation and Bandits via Additive Models (2015, methodology): “However, all such successes have been in low (typically 10 ) dimensions (Wang et al., 2013).”