Differentiating through the Fréchet Mean

By Aaron Lou, Isay Katsman, Qingxuan Jiang, Serge J. Belongie, Ser-Nam Lim, Christopher De Sa. 2020

87 citations recorded by the source corpus.

Recent advances in deep representation learning on Riemannian manifolds extend classical deep learning operations to better capture the geometry of the manifold. One possible extension is the Frechet mean, the generalization of the Euclidean mean; however, it has been difficult to apply because it lacks a closed form with an easily computable derivative. In this paper, we show how to differentiate through the Frechet mean for arbitrary Riemannian manifolds. Then, focusing on hyperbolic space, we derive explicit gradient expressions and a fast, accurate, and hyperparameter-free Frechet mean solver. This fully integrates the Frechet mean into the hyperbolic neural network pipeline. To demonstrate this integration, we present two case studies. First, we apply our Frechet mean to the existing Hyperbolic Graph Convolutional Network, replacing its projected aggregation to obtain state-of-the-art results on datasets with high hyperbolicity. Second, to demonstrate the Frechet mean's capacity to generalize Euclidean neural network operations, we develop a hyperbolic batch normalization method that gives an improvement parallel to the one observed in the Euclidean setting.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Geometry and Topology, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Geometry and Topology

Simon Lacoste-Julien

In Optimization & Training

Citations in the corpus

Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation (2021, methodology): “Beyond the sets and multisets considered in this work, implicit differentiation has been applied to other non-Euclidean domains, such as hyperbolic space (Lou et al., 2020) and graphs (Gu et al.”