Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry

By Maximilian Nickel, Douwe Kiela. International Conference on Machine Learning, 2018

604 citations recorded by the source corpus.

We are concerned with the discovery of hierarchical relationships from large-scale unstructured similarity scores. For this purpose, we study different models of hyperbolic space and find that learning embeddings in the Lorentz model is substantially more efficient than in the Poincar\'e-ball model. We show that the proposed approach allows us to learn high-quality embeddings of large taxonomies which yield improvements over Poincar\'e embeddings, especially in low dimensions. Lastly, we apply our model to discover hierarchies in two real-world datasets: we show that an embedding in hyperbolic space can reveal important aspects of a company's organizational structure as well as reveal historical relationships between language families.

Subfields: Artificial Intelligence, Molecular Biology, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/dc6826017bcb1f39f1bdf03c2c219e9fae804a17

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Statistical and Nonlinear Physics

Citations in the corpus

Hyperbolic Graph Neural Networks (2019, methodology): “Here, we compare standard graph convolutional networks [26] that work in Euclidean space with different hyperbolic graph neural networks (HGNNs): one that operates on the Poincaré ball as in [32] and one that operates on the Lorentz model of hyperbolic geometry as in [33].”
Multi-relational Poincaré Graph Embeddings (2019, methodology): “Certain knowledge graph relations, such as “hypernym” and “has_part”, induce a hierarchical structure over entities, suggesting that embedding them in hyperbolic rather than Euclidean space may lead to improved representations [26, 23, 24, 12, 30].”
Riemannian Adaptive Optimization Methods (2018, background): “…equations (Vander-eycken & Vandewalle, 2010), matrix factorization (Tan et al., 2014), geometric programming (Sra & Hosseini, 2015), dictionary learning (Cherian & Sra, 2017) or hyperbolic taxonomy embedding (Nickel & Kiela, 2017; Ganea et al., 2018a; De Sa et al., 2018; Nickel & Kiela, 2018).”
Machine Learning on Graphs: A Model and Comprehensive Taxonomy (2020, methodology): “2 Poincaré [83] Lorentz [84] Product [49]”
Differentiating through the Fréchet Mean (2020, methodology): “Recent advancements in geometric representation learning have utilized hyperbolic space for tree embedding tasks (Nickel & Kiela, 2017; 2018; Yu & De Sa, 2019).”
Poincaré GloVe: Hyperbolic Word Embeddings (2018, methodology): “…graph in a low dimensional space with the aim of improving link prediction either by imposing a partial order in the embedding space (Vendrov et al., 2015; Vilnis et al., 2018; Athiwaratkun & Wilson, 2018), by using hyperbolic geometry (Nickel & Kiela, 2017; 2018), or both (Ganea et al., 2018a).”
Hyperbolic Graph Convolutional Neural Networks (2019, methodology): “Empirically, both models give similar performance but hyperboloid model offers more stable optimization, because Poincaré distance is numerically unstable [30].”
Informing Unsupervised Pretraining with External Linguistic Knowledge (2019, background): “…vectors produced by these models produce substantial gains in a number of downstream tasks, e.g., in dialog state tracking (Mrkˇsi´c et al., 2017; Ren et al., 2018), text simpliﬁcation (Glavaˇs and Vuli´c, 2018; Saggion, 2017), and taxonomy induction (Nguyen et al., 2017; Nickel and Kiela, 2018).”
Geom-GCN: Geometric Graph Convolutional Networks (2020, background): “In the latent space, complicated topology patterns in graphs can be preserved and presented as intuitive geometry, such as subgraph (Narayanan et al., 2016), community (Ni et al., 2019), and hierarchy (Nickel & Kiela, 2017; 2018).”