GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation

By Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, Jian Tang. 2022

734 citations recorded by the source corpus.

Predicting molecular conformations from molecular graphs is a fundamental problem in cheminformatics and drug discovery. Recently, significant progress has been achieved with machine learning approaches, especially with deep generative models. Inspired by the diffusion process in classical non-equilibrium thermodynamics where heated particles will diffuse from original states to a noise distribution, in this paper, we propose a novel generative model named GeoDiff for molecular conformation prediction. GeoDiff treats each atom as a particle and learns to directly reverse the diffusion process (i.e., transforming from a noise distribution to stable conformations) as a Markov chain. Modeling such a generation process is however very challenging as the likelihood of conformations should be roto-translational invariant. We theoretically show that Markov chains evolving with equivariant Markov kernels can induce an invariant distribution by design, and further propose building blocks for the Markov kernels to preserve the desirable equivariance property. The whole framework can be efficiently trained in an end-to-end fashion by optimizing a weighted variational lower bound to the (conditional) likelihood. Experiments on multiple benchmarks show that GeoDiff is superior or comparable to existing state-of-the-art approaches, especially on large molecules.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Generative Models, Computational Chemistry & Drug Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

In Generative Models

In Computational Chemistry & Drug Discovery

Citations in the corpus

End-to-end protein–ligand complex structure generation with diffusion-based generative models (2022, methodology): “To ensure that the distributions are invariant to translation, we use distributions on the linear subspace [28] where the centroid of the molecular system is always at the origin and define Nx as a Gaussian distribution on this subspace, following [29].”
TANKBind: Trigonometry-Aware Neural NetworKs for Drug-Protein Binding Structure Prediction (2022, methodology): “We use Graph Isomorphism Network (GIN) Xu et al. [2018] to embed the compound and arrive at feature h c ∈ R m × s after graph propagation, where m is the number of heavy atoms and s is the embedding size.”
DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking (2022, methodology): “While DGMs have been applied to other problems in molecular machine learning [5, 6, 7], existing approaches are ill-suited for molecular docking, where the space of ligand poses is an ( m + 6) - dimensional submanifold M ⊂ R 3 n , where n and m are, respectively, the number of atoms and torsion…”
State-specific protein–ligand complex structure prediction with a multiscale deep generative model (2022, neutral): “Other studies 33–37 have shown that diffusion models are effective in modelling molecular structures beyond protein backbones, especially in the context of molecular docking and structure-based drug design.”
Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem (2022, positive): “[42] proved that if θ is equivariant to a group then pθ is invariant to the same group.”
Torsional Diffusion for Molecular Conformer Generation (2022, methodology): “A number of machine learning methods for conformer generation has been developed [Xu et al., 2021a,b, Shi et al., 2021, Luo et al., 2021], the most recent and advanced of which are GeoMol [Ganea et al., 2021] and GeoDiff [Xu et al., 2022].”
DiGress: Discrete Denoising diffusion for graph generation (2022, background): “Recent works have considered the discrete diffusion problem for text, image and audio data (Hooge-boom et al., 2021; Johnson et al., 2021; Yang et al., 2022).”
Uni-Mol: A Universal 3D Molecular Representation Learning Framework (2023, methodology): “We found the test sets used in GeoDiff [60] and DMCG [59] are slightly different from baselines (different filtering conditions), so we use the released model parameters of GeoDiff to reproduce the results on the same test set.”
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization (2023, methodology): “With the changing economic and geopolitical landscape within the transport and logistics industry, Last Mile Delivery (LMD) has become the most expensive portion of the logistics fulfillment chain, representing over 41% of overall supply chain costs [50].”
Structure-based drug design with equivariant diffusion models (2022, positive): “In order to circumvent this issue, we follow previous works (Köhler et al., 2020; Xu et al., 2022; Hoogeboom et al., 2022) by limiting the whole sampling process to a linear subspace where the center of mass (CoM) of the system is zero.”
Generating Molecular Conformer Fields (2023, methodology): “In GeoDiff (Xu et al., 2022), a diffusion model is used which focuses on crafting equivariant forward and backward processes with equivariant graph neural networks.”
Machine Learning for Synthetic Data Generation: a Review (2023, neutral): “…molecule distribution, including variational autoencoder (VAE) [21], [76], [77], generative adversarial network (GAN) [78], energy-based model (EBM) [79], [80], diffusion model [81], reinforcement learning (RL) [22], [24], [82], genetic algorithm [83], sampling-based methods [84], [85], etc.”
Crystal Diffusion Variational Autoencoder for Periodic Material Generation (2021, methodology): “Inspired by Xu et al. (2021a); Ganea et al. (2021), we deﬁne two coverage metrics, COV-R (Recall) and COV-P (Precision), to measure the similarity between ensembles of generated materials and ground truth materials in test set.”
On Distillation of Guided Diffusion Models (2022, methodology): “2 STEP-ONE DISTILLATION The model architecture we use is a U-Net model similar to the ones used in Ho & Salimans (2022). We use the same number of channels and attention as used in Ho & Salimans (2022) for both ImageNet 64x64 and CIFAR-10.”
DDGR: Continual Learning with Deep Diffusion-based Generative Replay (2023, background): “…have shown their exquisite potential in the field of computer vision (Dhariwal & Nichol, 2021; Ho et al., 2022b; Li et al., 2022; Luo & Hu, 2021), audio processing (Chen et al., 2021; Popov et al., 2021; Kim et al., 2022), and AI for science (Luo et al., 2021; Xu et al., 2022; Jing et al., 2022).”
Q-Diffusion: Quantizing Diffusion Models (2023, background): “With their flexibility on downstream tasks, diffusion models demonstrate their power in various applications such as image super-resolution [36, 23], inpainting [43], shape generation [5], graph generation [31], image-to-image translation [39], and molecular conformation generation [46].”
SE(3) diffusion model with application to protein backbone generation (2023, methodology): “6P# is the pushforward by P. posed for continuous normalizing flows and discrete time diffusion models (Köhler et al., 2020; Xu et al., 2022).”
MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “Two common metrics to evaluate the quality of the generated molecule conformations are Coverage (COV) and Matching (MAT) scores [170], [182]. where S g and S r denote generated and reference conformations, respectively. δ is a given RMSD threshold.”
DiffusionDet: Diffusion Model for Object Detection (2022, background): “…models have recently demonstrated remarkable results in fields including computer vision [4,21,33,35,39,68,71,75,76,82,104,108], nature language processing [3,30,52], audio processing [41,48,50,70, 91,100,103], graph-related topics [42], interdisciplinary applications [2,40,44,78,94,99,102], etc.”