Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space

By José Miguel Hernández-Lobato, James Requeima, Edward O. Pyzer-Knapp, Alán Aspuru-Guzik. 2017

206 citations recorded by the source corpus.

Chemical space is so large that brute force searches for new interesting molecules are infeasible. High-throughput virtual screening via computer cluster simulations can speed up the discovery process by collecting very large amounts of data in parallel, e.g., up to hundreds or thousands of parallel measurements. Bayesian optimization (BO) can produce additional acceleration by sequentially identifying the most useful simulations or experiments to be performed next. However, current BO methods cannot scale to the large numbers of parallel measurements and the massive libraries of molecules currently used in high-throughput screening. Here, we propose a scalable solution based on a parallel and distributed implementation of Thompson sampling (PDTS). We show that, in small scale problems, PDTS performs similarly as parallel expected improvement (EI), a batch version of the most widely used BO heuristic. Additionally, in settings where parallel EI does not scale, PDTS outperforms other scalable baselines such as a greedy search, $\epsilon$-greedy approaches and a random search method. These results show that PDTS is a successful solution for large-scale parallel BO.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Management Science and Operations Research

Citations in the corpus

Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020 (2021, methodology): “For example, chemical engineering, materials discovery, manufacturing design, control systems and drug discovery are also effective uses of BO (Hernández-Lobato et al., 2017; Ueno et al., 2016; Frazier and Wang, 2016; Negoescu et al., 2011; Ju et al., 2017; Haghanifar et al., 2019; Candelieri et al.”
Recent Advances in Bayesian Optimization (2022, background): “Thompson sampling can be fully distributed and handle the asynchronously parallel setting [75], although it fails to perform well due to its inherent randomness.”
Bayesian Optimization for Categorical and Category-Specific Continuous Inputs (2019, unknown): “Recently, Thompson sampling has become an efficient technique for batch BO (Hernández-Lobato et al. 2017; Kandasamy et al. 2018); the idea is to select a batch element by maximizing a randomly drawn function from the posterior GP.”
Decentralized Exploration in Multi-Armed Bandits (2018, methodology): “In Hernandez-Lobato et al (2017), the best arm identiﬁcation task with ﬁxed budget is distributed using Thompson Sampling in order to accelerate the exploration of the chemical space.”
Constrained Bayesian Optimization with Noisy Experiments (2017, methodology): “Recently the classic Thompson sampling algorithm (Thompson, 1933) has been applied to GP Bayesian optimization (Hernández-Lobato et al., 2017; Kandasamy et al., 2018).”
Asynchronous Parallel Bayesian Optimisation via Thompson Sampling (2017, methodology): “We mention that parallelised TS has been explored to varying degrees in some applied domains of bandit and reinforcement learning research (Israelsen et al., 2017; Hernández-Lobato et al., 2017; Osband et al., 2016).”
Scalable Global Optimization via Local Bayesian Optimization (2019, methodology): “Although originally developed for bandit problems [39], it has recently shown its value in BO [20, 4, 26].”
Parallelised Bayesian Optimisation via Thompson Sampling (2018, background): “We mention that parallelised versions of TS have been explored to varying degrees in some applied domains of bandit and reinforcement learning research [15, 18, 31].”
Actively Learning what makes a Discrete Sequence Valid (2017, methodology): “This is akin to Thompson sampling (TS), commonly used in Bayesian optimisation ( Hern´andez-Lobato et al., 2017) and reinforcement learning (Chapelle & Li, 2011) to balance exploration and exploitation.”
Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search (2020, methodology): “We select to compare our k-DPP quality against Thompson sampling (Hernández-Lobato et al., 2017), GP-BUCB (Desautels et al., 2014) and k-DPP for batch BO (Kathuria et al., 2016).”
Machine-learned potentials for next-generation matter simulations (2021, methodology): “Active learning approaches have furthermore been applied to accelerate chemical-space exploration (based on Thompson sampling 97 ), to identify suitable collective variables (based on reinforcement learning 98 ), as well as for the prediction of molecular properties 99 (for example, atomization…”
Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start (2017, methodology): “We compare to random Fourier basis expansions [17] as they have already been successfully applied to BO [11, 12].”
Bayesian reaction optimization as a tool for chemical synthesis (2021, background): “…as a Gaussian process represents a distribution over functions, one can draw and maximize candidate models that fit the data where information is available and vary according to function shape and estimated variance in areas where little is known—a process called Thompson sampling 39,40 .”