OpenFold: retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

By Gustaf Ahdritz, Nazim Bouatta, Sachin Kadyan, Qinghui Xia, William Gerecke, Tim O’Donnell, Daniel Berenberg, Ian Fisk, Niccoló Zanichelli, Bo Zhang and 18 more. 2023

435 citations recorded by the source corpus.

AlphaFold2 revolutionized structural biology with the ability to predict protein structures with exceptionally high accuracy. Its implementation, however, lacks the code and data required to train new models. These are necessary to (i) tackle new tasks, like protein-ligand complex structure prediction, (ii) investigate the process by which the model learns, which remains poorly understood, and (iii) assess the model’s generalization capacity to unseen regions of fold space. Here we report OpenFold, a fast, memory-efficient, and trainable implementation of AlphaFold2. We train OpenFold from scratch, fully matching the accuracy of AlphaFold2. Having established parity, we assess OpenFold’s capacity to generalize across fold space by retraining it using carefully designed datasets. We find that OpenFold is remarkably robust at generalizing despite extreme reductions in training set size and diversity, including near-complete elisions of classes of secondary structure elements. By analyzing intermediate structures produced by OpenFold during training, we also gain surprising insights into the manner in which the model learns to fold proteins, discovering that spatial dimensions are learned sequentially. Taken together, our studies demonstrate the power and utility of OpenFold, which we believe will prove to be a crucial new resource for the protein modeling community.

Subfields: Molecular Biology, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Evaluation & Benchmarks

Citations in the corpus

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, background): “…made big splashes as generative models in areas as diverse as text-to-image synthesis (Ramesh et al., 2022; Crowson et al., 2022; Rombach et al., 2022), protein modeling (Jumper et al., 2021; Ahdritz et al., 2022), and computer programming (Chen et al., 2021; Xu et al., 2022; Fried et al., 2022).”
End-to-end protein–ligand complex structure generation with diffusion-based generative models (2022, methodology): “These methods are regression-based and rely on ad hoc techniques like MSA subsampling, changing the number of recycles, and enabling dropout during inference to sample multiple structures [44].”
SE(3) diffusion model with application to protein backbone generation (2023, methodology): “Our implementation is based on OpenFold (Ahdritz et al., 2022) which is not compatible with efficient batching strategies implemented in PyTorch Geometric (Fey & Lenssen, 2019).”
Boltz-1 Democratizing Biomolecular Interaction Modeling (2024, positive): “We follow the OpenFold chunking implementation for the Triangular Attention layer [Ahdritz et al., 2024], and extend it to the MSA Module, applying it at three critical points: the transition layers, pair-weighted average layers, and outer product layers.”
Emergent and Predictable Memorization in Large Language Models (2023, background): “…state-of-the-art results outside of NLP on diverse tasks such as text-to-image synthesis [Ramesh et al., 2022, Crowson et al., 2022, Rombach et al., 2022], code generation [Chen et al., 2021, Xu et al., 2022, Fried et al., 2022], and protein modeling [Jumper et al., 2021, Ahdritz et al., 2022].”