Evolutionary velocity with protein language models

By Brian L. Hie, Kevin Kaichuang Yang, Peter S. Kim. 2021

151 citations recorded by the source corpus.

Predicting the order of biological homologs is a fundamental task in evolutionary biology. For protein evolution, this order is often determined by first arranging sequences into a phylogenetic tree, which has limiting assumptions and can suffer from substantial ambiguity. Here, we demonstrate how machine learning algorithms called language models can learn mutational likelihoods that predict the directionality of evolution, thereby enabling phylogenetic analysis that addresses key limitations of existing methods. Our main conceptual advance is to construct a “vector field” of protein evolution through local evolutionary predictions that we refer to as evolutionary velocity (evo-velocity). We show that evo-velocity can successfully predict evolutionary order at vastly different timescales, from viral proteins evolving over years to eukaryotic proteins evolving over geologic eons. Evo-velocity also yields new evolutionary insights, predicting strategies of viral-host immune escape, resolving conflicting theories on the evolution of serpins, and revealing a key role of horizontal gene transfer in the evolution of eukaryotic glycolysis. In doing so, our work suggests that language models can learn sufficient rules of natural protein evolution to enable evolutionary predictability.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Evolutionary-scale prediction of atomic level protein structure with a language model (2022, background): “[55] Brian L Hie, Kevin K Yang, and Peter S Kim.”
Exploring evolution-based & -free protein language models as protein function predictors (2022, background): “Recent work revealed that PLMs are strong zero-shot learners in predicting potential viral mutations [16, 17].”
Comprehensive phylogenetic analysis of the ribonucleotide reductase family reveals an ancestral clade (2022, unknown): “Protein NLP models can provide valuable and phylogeny-independent evolutionary insight ( Alley et al., 2019 ; Hie et al., 2021 ; Rives et al., 2021 ).”
Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, background): “…protein MLMs contain structural information (Rao et al., 2019; Rives et al., 2021; Chowdhury et al., 2021), encode evolutionary trajectories (Hie et al., 2022b; 2021), are zero-shot predictors of mutation fitness effects (Meier et al., 2021), improve out-of-domain generalization on protein…”
Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies (2022, methodology): “Embeddings from transformer encoder models trained for masked language modeling have been used for variant prediction 20 , evolutionary analysis 21,22 , and as features for protein structure prediction 23,24 .”
Analysis of insertions and extensions in the functional evolution of the ribonucleotide reductase family (2022, methodology): “Evo-velocity utilizes language models to infer the fitness of individual protein sequences as well as the evolutionary direction without the construction of a phylogenetic tree.(28) We previously applied evo-velocity analysis on the full RNR sequences to confirm our interpretation of the RNR phyloge”
Efficient evolution of human antibodies from general protein language models (2023, unknown): “More broadly, a major open question 10 is whether general evolutionary information (for example, learning patterns from sequence variation across past evolution) is sufficient to enable efficient evolution under specific selection pressures (for example, higher binding affinity to a specific…”