Normalizing Flows on Riemannian Manifolds

By Mevlana Gemici, Danilo Jimenez Rezende, Shakir Mohamed. 2016

120 citations recorded by the source corpus.

We consider the problem of density estimation on Riemannian manifolds. Density estimation on manifolds has many applications in fluid-mechanics, optics and plasma physics and it appears often when dealing with angular variables (such as used in protein folding, robot limbs, gene-expression) and in general directional statistics. In spite of the multitude of algorithms available for density estimation in the Euclidean spaces $\mathbf{R}^n$ that scale to large n (e.g. normalizing flows, kernel methods and variational approximations), most of these methods are not immediately suitable for density estimation in more general Riemannian manifolds. We revisit techniques related to homeomorphisms from differential geometry for projecting densities to sub-manifolds and use it to generalize the idea of normalizing flows to more general Riemannian manifolds. The resulting algorithm is scalable, simple to implement and suitable for use with automatic differentiation. We demonstrate concrete examples of this method on the n-sphere $\mathbf{S}^n$.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Equivariant Hamiltonian Flows (2019, background): “But it remains a challenge to embed flow-based models with domain knowledge such as data on specific manifolds [6] and known symmetries or factorisation of the target density.”
Neural Autoregressive Flows (2018, background): “There Finally, there are also several works which develop new techniques for constructing NFs that are orthogonal to ours (Tomczak & Welling, 2017; 2016; Gemici et al., 2016; Du-venaud et al., 2016; Berg et al., 2018).”
Refining Deep Generative Models via Discriminator Gradient Flow (2020, unknown): “If Jg has full column rank and g is an injective function, then we have the following change-of-variables equation (Ben-Israel, 1999; Gemici et al., 2016), ∫ X f(x)dx = ∫ Z (f ◦ g)(z) √ detJ>g Jg(z)dz.”
Hyperspherical Variational Auto-Encoders (2018, background): “In (Gemici et al., 2016) a first attempt is made to extend normalizing flows to Riemannian manifolds.”
Normalizing Flows: An Introduction and Review of Current Methods (2020, background): “Modelling probability distributions on manifolds has applications in many fields including robotics, molecular biology, optics, fluid mechanics, and plasma physics [30], [79].”
Riemannian Score-Based Generative Modeling (2022, methodology): “Additionally, we consider a standard SGM on the 2D plane followed by the inverse stereographic projection which induces a density on the sphere (Gemici et al., 2016).”