Bayesian Optimization over Sets

By Jungtaek Kim, M. McCourt, Tackgeun You, Saehoon Kim, Seungjin Choi. 2019

9 citations recorded by the source corpus.

We propose a Bayesian optimization method over sets, to minimize a black-box function that can take a set as single input. Because set inputs are permutation-invariant and variable-length, traditional Gaussian process-based Bayesian optimization strategies which assume vector inputs can fall short. To address this, we develop a Bayesian optimization method with \emph{set kernel} that is used to build surrogate functions. This kernel accumulates similarity over set elements to enforce permutation-invariance and permit sets of variable size, but this comes at a greater computational cost. To reduce this burden, we propose a more efficient probabilistic approximation which we prove is still positive definite and is an unbiased estimator of the true set kernel. Finally, we present several numerical experiments which demonstrate that our method outperforms other methods in various applications.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “Despite being a proven technique for automated algorithm configuration tasks [25–27], the state-of-the-art of BO does not reconcile well with variable-length solution problems [29,30].”
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining (2020, background): “While recent works have tried to develop models to extend BO to either structured [1, 27, 6, 42] or high-dimensional [25, 39, 18] input spaces, we are not aware of any BO method that can efﬁciently handle the types of high-dimensional and structured spaces considered in this work.”