FLIP: Benchmark tasks in fitness landscape inference for proteins

By Christian Dallago, Jody Mou, Kadina E. Johnston, Bruce J. Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, Kevin Kaichuang Yang. 2021

190 citations recorded by the source corpus.

Machine learning could enable an unprecedented level of control in protein engineering for therapeutic and industrial applications. Critical to its use in designing proteins with desired properties, machine learning models must capture the protein sequence-function relationship, often termed fitness landscape. Existing bench-marks like CASP or CAFA assess structure and function predictions of proteins, respectively, yet they do not target metrics relevant for protein engineering. In this work, we introduce Fitness Landscape Inference for Proteins (FLIP), a benchmark for function prediction to encourage rapid scoring of representation learning for protein engineering. Our curated tasks, baselines, and metrics probe model generalization in settings relevant for protein engineering, e.g. low-resource and extrapolative. Currently, FLIP encompasses experimental data across adeno-associated virus stability for gene therapy, protein domain B1 stability and immunoglobulin binding, and thermostability from multiple protein families. In order to enable ease of use and future expansion to new tasks, all data are presented in a standard format. FLIP scripts and data are freely accessible at https://benchmark.protein.properties.

Subfields: Genetics, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

Learning inverse folding from millions of predicted structures (2022, methodology): “We report the zero-shot performance on each of the 7 data subsets evaluated in the FLIP (Dallago et al., 2021) benchmark suite.”
Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, methodology): “For example, this prevented us from evaluating on the FLIP AAV landscape.”
ProGen2: Exploring the Boundaries of Protein Language Models (2022, background): “We refer the reader to (Dallago et al., 2021; Yang et al., 2022) for further discussion.”
Bilingual language model for protein sequence and structure (2024, unknown): “data [38], [81], which is why we focused on a limited number of standard benchmarks that we tried to reproduce as closely as possible to existing work using biotrainer [36] and FLIP [37].”
Exploring evolution-based & -free protein language models as protein function predictors (2022, methodology): “In this paper, we focus on functional annotation prediction (classiﬁcation task) [45, 5] and ﬁtness prediction (including both regression [8, 25] and zero-shot prediction [16, 24] tasks), as shown in Figure 2.”