Masked Inverse Folding with Sequence Transfer for Protein Representation Learning

By Kevin Kaichuang Yang, Niccoló Zanichelli, Hugh Yeh. 2023

129 citations recorded by the source corpus.

Self-supervised pretraining on protein sequences has led to state-of-the art performance on protein function and fitness prediction. However, sequence-only methods ignore the rich information contained in experimental and predicted protein structures. Meanwhile, inverse folding methods reconstruct a protein’s amino-acid sequence given its structure, but do not take advantage of sequences that do not have known structures. In this study, we train a masked inverse folding protein masked language model parameterized as a structured graph neural network. During pretraining, this model learns to reconstruct corrupted sequences conditioned on the backbone structure. We then show that using the outputs from a pretrained sequence-only protein masked language model as input to the inverse folding model further improves pretraining perplexity. We evaluate both of these models on downstream protein engineering tasks and analyze the effect of using information from experimental or predicted structures on performance.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

1. Aurelie C. Lozano
2. Zuobai Zhang
3. Jian Tang
4. Payel Das
5. Vijil Chenthamarakshan
6. Minghao Xu
7. Arian Jamasb

Citations in the corpus

• Protein Representation Learning by Geometric Structure Pretraining (2022, methodology): “, 2019) and Masked Inverse Folding in the protein community (Yang et al., 2022).”