ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning

By Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, G. Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas B. Fehér, Christoph Angerer, Martin Steinegger and 2 more. 2021

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Kevin Kaichuang Yang
2. Surojit Biswas
3. Ratul Chowdhury
4. Nazim Bouatta
5. George M. Church
6. Mohammed Alquraishi
7. Gustaf Ahdritz
8. Charlotte Rochereau

Citations in the corpus

• Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, background): “…protein language models have advanced the ability of machine learning models to predict protein function and fitness from sequence, especially when labeled training data is scarce Alley et al. (2019); Rives et al. (2021); Rao et al. (2019; 2020); Elnaggar et al. (2021); Brandes et al. (2021).”
• Single-sequence protein structure prediction using language models and deep learning (2022, methodology): “First, 2–8 contiguous residues were masked simultaneously in each sequence (similar to the ProtTrans 37 language model), making the reconstruction task harder and emphasizing learning from global, rather than local, context.”