The language of proteins: NLP, machine learning & protein sequences

By Dan Ofer, Nadav Brandes, Michal Linial. Computational and Structural Biotechnology Journal, 2021

336 citations recorded by the source corpus.

Natural language processing (NLP) is a field of computer science concerned with automated text and language analysis. In recent years, following a series of breakthroughs in deep and machine learning, NLP methods have shown overwhelming progress. Here, we review the success, promise and pitfalls of applying NLP algorithms to the study of proteins. Proteins, which can be represented as strings of amino-acid letters, are a natural fit to many NLP methods. We explore the conceptual similarities and differences between proteins and language, and review a range of protein-related tasks amenable to machine learning. We present methods for encoding the information of proteins as text and analyzing it with NLP methods, reviewing classic concepts such as bag-of-words, k-mers/n-grams and text search, as well as modern techniques such as word embedding, contextualized embedding, deep learning and neural language models. In particular, we focus on recent innovations such as masked language modeling, self-supervised learning and attention-based models. Finally, we discuss trends and challenges in the intersection of NLP and protein research.

Subfields: Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/ef25b02f3be31c699255ee05aa90a4a17461d95d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Protein language model embeddings for fast, accurate, alignment-free protein structure prediction (2021, methodology): “…implicitly learning grammar, pLM embeddings decode some aspects of the language of life as written in protein sequences (Heinzinger et al., 2019; Ofer et al., 2021) which suffices as exclusive input to many methods predicting aspects of protein structure and function without any further…”
Genome-wide prediction of disease variants with a deep protein language model (2022, methodology): “A different deep-learning approach to functionalizing variants uses protein language models, a technique originally developed for natural language processing, which has been successfully applied to many protein prediction tasks [18–23].”
ProteinBERT: a universal deep-learning model of protein sequence and function (2021, background): “As such, machinelearning methods developed for natural language and other sequences are a natural fit to predictive protein tasks [1].”