Equivariant Flows: exact likelihood generative learning for symmetric densities

By Jonas Köhler, Leon Klein, Frank No'e. 2020

351 citations recorded by the source corpus.

Normalizing flows are exact-likelihood generative neural networks which approximately transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry. To scale and generalize these results, it is essential that the natural symmetries in the probability density - in physics defined by the invariances of the target potential - are built into the flow. We provide a theoretical sufficient criterion showing that the distribution generated by equivariant normalizing flows is invariant with respect to these symmetries by design. Furthermore, we propose building blocks for flows which preserve symmetries which are usually found in physical/chemical many-body particle systems. Using benchmark systems motivated from molecular physics, we demonstrate that those symmetry preserving flows can provide better generalization capabilities and sampling efficiency.

Subfields: Artificial Intelligence, Materials Chemistry, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Materials Chemistry

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

E(n) Equivariant Graph Neural Networks (2021, neutral): “Recently, various forms and methods to achieve E(3) or SE(3) equivariance have been proposed (Thomas et al., 2018; Fuchs et al., 2020; Finzi et al., 2020; Köhler et al., 2020).”
GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation (2022, methodology): “It has shown effectiveness to integrate such inductive bias into model parameterization for modeling 3D geometry, which is critical for the generalization capacity (Köhler et al., 2020; Satorras et al., 2021a).”
Learning Gradient Fields for Molecular Conformation Generation (2021, background): “Endowing model with such inductive biases is critical for better generalization and successful learning (Köhler et al., 2020).”
Generating Molecular Conformer Fields (2023, background): “In particular, enforcing equivariance as an explicit inductive bias in neural networks can often lead to improved generalization (K¨ohler et al., 2020) by constraining the space of functions that can be represented by a model.”
Inverse design of 3d molecular structures with conditional generative neural networks (2021, background): “This includes specifically designed approaches to translate given molecular graphs to 3d conformations [32–38], map from coarse-grained to fine-grained structures [39], sample unbiased equilibrium configurations of a given system [40, 41], or focus on protein folding [42–46].”
3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction (2023, background): “) should be invariant to translation and rotation of the protein-ligand complex, which is a critical inductive bias for generating 3D objects such as molecules (K¨ohler et al., 2020; Satorras et al., 2021a; Xu et al., 2022; Hoogeboom et al., 2022).”
Equivariant Diffusion for Molecule Generation in 3D (2022, background): “…(Simm & Hern´andez-Lobato, 2019), Wasser-stein GANs (Hoffmann & No´e, 2019), and normalizing ﬂows (No´e et al., 2019), with adaptions for Euclidean symmetries in (K¨ohler et al., 2020; Xu et al., 2021a; Simm et al., 2021; Ganea et al., 2021; Guan et al., 2022) resulting in performance improvements.”
SE(3) diffusion model with application to protein backbone generation (2023, methodology): “6P# is the pushforward by P. posed for continuous normalizing flows and discrete time diffusion models (Köhler et al., 2020; Xu et al., 2022).”
Structure-based drug design with equivariant diffusion models (2022, neutral): “In order to circumvent this issue, we follow previous works (Köhler et al., 2020; Xu et al., 2022; Hoogeboom et al., 2022) by limiting the whole sampling process to a linear subspace where the center of mass (CoM) of the system is zero.”