Universal Transforming Geometric Network

By Jin Li. 2019

9 citations recorded by the source corpus.

The recurrent geometric network (RGN), the first end-to-end differentiable neural architecture for protein structure prediction, is a competitive alternative to existing models. However, the RGN's use of recurrent neural networks (RNNs) as internal representations results in long training time and unstable gradients. And because of its sequential nature, it is less effective at learning global dependencies among amino acids than existing transformer architectures. We propose the Universal Transforming Geometric Network (UTGN), an end-to-end differentiable model that uses the encoder portion of the Universal Transformer architecture as an alternative for internal representations. Our experiments show that compared to RGN, UTGN achieve a $1.7$ \si{\angstrom} improvement on the free modeling portion and a $0.7$ \si{\angstrom} improvement on the template based modeling of the CASP12 competition.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Highly accurate protein structure prediction with AlphaFold (2021, critical): “A few recent studies have been developed to predict the 3D coordinates directly 47–50 , but the accuracy of these approaches does not match traditional, hand-crafted structure prediction pipelines 51 .”
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers (2020, background): “Machine Translation (NMT) [6], document generation/summarization, time series prediction, generative modeling (e.g. image generation [36]), music generation [26], and analysis of biological sequences [39, 33, 30]. Transformers rely on a trainable attention mechanism that speciﬁes complex dependencie”