Learning from Protein Structure with Geometric Vector Perceptrons

By Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael J. L. Townshend, Ron O. Dror. 2020

664 citations recorded by the source corpus.

Learning on 3D structures of large biomolecules is emerging as a distinct area in machine learning, but there has yet to emerge a unifying network architecture that simultaneously leverages the graph-structured and geometric aspects of the problem domain. To address this gap, we introduce geometric vector perceptrons, which extend standard dense layers to operate on collections of Euclidean vectors. Graph neural networks equipped with such layers are able to perform both geometric and relational reasoning on efficient and natural representations of macromolecular structure. We demonstrate our approach on two important problems in learning from protein structure: model quality assessment and computational protein design. Our approach improves over existing classes of architectures, including state-of-the-art graph-based and voxel-based methods.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Model Architectures, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

In Model Architectures

In Computational Biology & Bioinformatics

Citations in the corpus

TANKBind: Trigonometry-Aware Neural NetworKs for Drug-Protein Binding Structure Prediction (2022, methodology): “We then apply the geometric vector perceptrons (GVP) [18, 37] to embed the protein and arrive at feature h ∈ Rn×s after graph propagation, where n is the number of nodes and s is the embedding size.”
State-specific protein–ligand complex structure prediction with a multiscale deep generative model (2022, positive): “2e and Supplementary Information Section E) is inspired by previous works on 3D graph and attentional neural networks for point clouds 61,62 , rigid-body simulations 63 and biopolymer representation learning 1,64–66 .”
EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction (2022, methodology): “Various other protein representations have been proposed for (graph) DL methods for individual structure prediction (Jing et al., 2020), protein-protein interactions (Dai & Bailey-Kellogg, 2021; Eismann et al., 2020; Townshend et al., 2019), or protein function prediction (Gligorijevi´c et al.,…”
Learning inverse folding from millions of predicted structures (2022, methodology): “We compare zero-shot predictions using inverse folding models to supervised and transfer learning methods (Townshend et al., 2020; Jing et al., 2021a) on the Atom3D test set.”
GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation (2022, background): “The 3D structures determine the biological and physical properties of molecules and hence play a key role in many applications such as computational drug and material de-sign (Thomas et al., 2018; Gebauer et al., 2021; Jing et al., 2021; Batzner et al., 2021).”
Torsional Diffusion for Molecular Conformer Generation (2022, background): “While there exist several GNN architectures which are SE (3) -equivariant [Jing et al., 2021, Satorras et al., 2021], their SE (3) -invariant outputs are also parity invariant and, therefore, cannot satisfy the desired symmetry.”
Equivariant message passing for the prediction of tensorial properties and molecular spectra (2021, background): “, 2018b) and show similarities to geometric vector perceptrons (GVP) (Jing et al., 2021).”
Protein Representation Learning by Geometric Structure Pretraining (2022, methodology): “GVP (Jing et al., 2021).”
Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets (2022, neutral): “The formula of the original geometric vector perceptron Here we briefly formulize the original geometric geometric vector perceptron (GVP) and the details can be found in Jing et al. (2020).”
Dynamic-Backbone Protein-Ligand Structure Prediction with Multiscale Generative Diffusion Models (2022, methodology): “The architecture of ESDM (Figure1e) is inspired by prior works on 3D graph and attentional neural networks for point clouds [37, 38], rigid-body simulations [39] and biopolymer representation learning [9, 40–42].”
Illuminating protein space with a programmable generative model (2022, background): “7, 8 and 15), but with conditional sequence and side-chain decoding layers that build on previous studies 15,16 that have seen further refinement and experimental validation 32–34 .”
Masked Inverse Folding with Sequence Transfer for Protein Representation Learning (2023, methodology): “The GVP architecture improves pretraining perplexity and recovery, but we find that it does not improve performance on downstream tasks.”
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs (2022, background): “Equivariant neural networks [72, 43, 80, 23, 50, 74, 4, 38, 65, 63, 77, 5, 71, 46, 51] operate on geometric tensors like type-L vectors to achieve equivariance.”
Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds (2023, background): “, 2021) and GVP (Jing et al., 2021), provide powerful tools for geometric reasoning in an SE(3)-equivariant manner.”
Predicting the locations of cryptic pockets from single protein structures using the PocketMiner graph neural network (2022, methodology): “We trained GVP-GNN models using the architecture described above with the following hyperparameters: • Structural feature node dimensions: 8 vector dimensions, 50 scalar dimensions • Structural feature edge dimensions: 1 vector dimension, 32 scalar dimensions • GVP hidden dimensions: 16 vector dimen”
ProGen2: Exploring the Boundaries of Protein Language Models (2022, methodology): “, 2022) replaced the encoder network with an rotation-equivariant model (Jing et al., 2020) and scaled the network to 142M parameters, observing further improvements to sequence recovery with increased model capacity.”
Equivariant Graph Neural Networks for 3D Macromolecular Structure (2021, neutral): “Schematic of the original geometric vector perceptron (GVP) as described in Jing et al. (2021) ( top ) and the modiﬁed GVP presented in Algorithm 1 ( bottom ).”
Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures (2022, background): “The joint distribution of a CDR sequence and its structure is conditional on the antigen structures which are modeled by equivariant neural networks [25, 24].”