Hyperbolic Neural Networks
By Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann. Neural Information Processing Systems, 2018
842 citations recorded by the source corpus.
Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean geometry, mostly because of the absence of corresponding hyperbolic neural network layers. This makes it hard to use hyperbolic embeddings in downstream tasks. Here, we bridge this gap in a principled manner by combining the formalism of Mobius gyrovector spaces with the Riemannian geometry of the Poincare model of hyperbolic spaces. As a result, we derive hyperbolic versions of important deep learning tools: multinomial logistic regression, feed-forward and recurrent neural networks such as gated recurrent units. This allows to embed sequential data and perform classification in the hyperbolic space. Empirically, we show that, even if hyperbolic optimization tools are limited, hyperbolic sentence embeddings either outperform or are on par with their Euclidean variants on textual entailment and noisy-prefix recognition tasks.
Subfields: Artificial Intelligence, Computational Mathematics, Computational Theory and Mathematics
View Paper: https://www.semanticscholar.org/paper/562fa7faef812294bb1f235c97584f8560fb5cc0
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Christopher R'e
- Yoshua Bengio
- Timothy M. Hospedales
- Quoc V. Le
- Jure Leskovec
- Percy Liang
- Sergey Levine
- Mohit Bansal
In Computational Mathematics
- Timothy M. Hospedales
- Ivana Balazevic
- Carl Allen
- Octavian-Eugen Ganea
- Gary Bécigneul
- A. Tifrea
- Christopher R'e
- Jure Leskovec
In Computational Theory and Mathematics
- Timothy M. Hospedales
- Gary Bécigneul
- Octavian-Eugen Ganea
- Rex Ying
- Ivana Balazevic
- Carl Allen
- A. Tifrea
- Frank No'e
Citations in the corpus
- Hyperbolic Image Embeddings (2019, methodology): “Similarly to the paper [11], we use an additional hyperparameter c which modifies the curvature of Poincaré ball; it is then defined as Dc = {x ∈ R : c‖x‖2 < 1, c ≥ 0}.”
- Hyperspherical Prototype Networks (2019, neutral): “Several works have investigated prior and semantic knowledge in hyperbolic spaces [8, 32, 40].”
- Hyperbolic Graph Convolutional Neural Networks (2019, methodology): “In order to perform bias addition, we use a result from the HNN model and define b as an Euclidean vector located at T o H d,K .”
- Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020, background): “Instead, M¨obius addition ⊕ c (Ganea et al., 2018) provides an analogue to Euclidean addition for hyperbolic space.”
- Poincaré GloVe: Hyperbolic Word Embeddings (2018, methodology): “Finally, these word vectors can also be used in downstream tasks as explained by Ganea et al. (2018b).”
- Riemannian Adaptive Optimization Methods (2018, methodology): “• Distance function and geodesics, (Nickel & Kiela, 2017; Ungar, 2008; Ganea et al., 2018b).”
- Multi-relational Poincaré Graph Embeddings (2019, methodology): “[11] introduced Hyperbolic Neural Networks, connecting hyperbolic geometry with deep learning.”
- Differentiating through the Fréchet Mean (2020, methodology): “Introduced in Ganea et al. (2018), hyperbolic neural networks provide a natural generalization of standard neural networks.”
- Artificial Neural Networks and Machine Learning – ICANN 2021: 30th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part III (2021, background): “Here, basic manifold-preserving operations such as addition and multiplication in the context of neural networks have been extended to hyperbolic geometry [10].”
- HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation (2026, unknown): “However, this approach provably contracts representations toward the origin (Ganea et al., 2018a; Chami et al., 2019), destroying hierarchical structure encoded in radial depth.”