Conditioning by adaptive sampling for robust design

By David H. Brookes, Hahnbeom Park, Jennifer Listgarten. 2019

236 citations recorded by the source corpus.

We present a new method for design problems wherein the goal is to maximize or specify the value of one or more properties of interest. For example, in protein design, one may wish to find the protein sequence that maximizes fluorescence. We assume access to one or more, potentially black box, stochastic "oracle" predictive functions, each of which maps from input (e.g., protein sequences) design space to a distribution over a property of interest (e.g. protein fluorescence). At first glance, this problem can be framed as one of optimizing the oracle(s) with respect to the input. However, many state-of-the-art predictive models, such as neural networks, are known to suffer from pathologies, especially for data far from the training distribution. Thus we need to modulate the optimization of the oracle inputs with prior knowledge about what makes `realistic' inputs (e.g., proteins that stably fold). Herein, we propose a new method to solve this problem, Conditioning by Adaptive Sampling, which yields state-of-the-art results on a protein fluorescence problem, as compared to other recently published approaches. Formally, our method achieves its success by using model-based adaptive sampling to estimate the conditional distribution of the input sequences given the desired properties.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Statistics and Probability, Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Statistics and Probability

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

Generating functional protein variants with variational autoencoders (2020, methodology): “design applications because it can straightforwardly be used to bias generation towards particular 108 regions of sequence space, either by sampling from the vicinity of the latent representations of 109 target sequences, or by facilitating optimization based strategies which search the latent space”
Unified rational protein engineering with sequence-based deep representation learning (2019, background): “14), evoking deep protein design, similar to previous work with small molecules 48 and proteins 43 .”
Learning inverse folding from millions of predicted structures (2022, methodology): “Additionally, as an alternative to sequence generation models, model-guided algorithms design sequences based on predictive models as oracles (Yang et al., 2019; Angermueller et al., 2019; Brookes et al., 2019; Sinai et al., 2020).”
Guided Generative Protein Design using Regularized Transformers (2022, methodology): “This method was further developed to include weighting candidates by the uncertainty of generative model and the predictive model used for ﬁtness estimation Brookes et al. [2019].”
Explainable Machine Learning for Scientific Insights and Discoveries (2019, background): “In this way, it is possible to generate new synthetic data samples that optimize the value of the quantity or property by leveraging an adaptive sampling technique over the generative model; see also [48].”
Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection (2019, background): “…space Z too far away from the training data, which might yield latent points z that decode to semantically meaningless and/or syntactically invalid inputs x (Kusner et al., 2017; Grifﬁths & Hern´andez-Lobato, 2017; Brookes et al., 2019; Mahmood & Hern´andez-Lobato, 2019; Alperstein et al., 2019).”
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining (2020, methodology): “Indeed, particular instantiations of the CE method such as reward-weighted regression [53], feedback GAN [24], and design/conditioning by adaptive sampling (DbAS/CbAS) [8, 7] have been applied to similar problem settings as our work.”
Protein sequence design with deep generative models (2021, methodology): “An emerging approach to this optimization problem is to optimize with generative models [38, 39, 40].”
Protein design and variant prediction using autoregressive generative models (2021, background): “Previous neural language models 54 – 56 are so far not suitable for mutation effect prediction for sequences without extensive experimental data or for sequences with high variabilities, such as the CDRs of antibody variable domains.”