Hyperbolic Entailment Cones for Learning Hierarchical Embeddings

By Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann. International Conference on Machine Learning, 2018

360 citations recorded by the source corpus.

Learning graph representations via low-dimensional embeddings that preserve relevant network properties is an important class of problems in machine learning. We here present a novel method to embed directed acyclic graphs. Following prior work, we first advocate for using hyperbolic spaces which provably model tree-like structures better than Euclidean geometry. Second, we view hierarchical relations as partial orders defined using a family of nested geodesically convex cones. We prove that these entailment cones admit an optimal shape with a closed form expression both in the Euclidean and hyperbolic spaces. Moreover, they canonically define the embedding learning process. Experiments show significant improvements of our method over strong recent baselines both in terms of representational capacity and generalization.

View Paper: https://www.semanticscholar.org/paper/a4e088a3d92f9299b6a8060c21a1082ecb251902

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Multi-relational Poincaré Graph Embeddings (2019, background): “Certain knowledge graph relations, such as “hypernym” and “has_part”, induce a hierarchical structure over entities, suggesting that embedding them in hyperbolic rather than Euclidean space may lead to improved representations [26, 23, 24, 12, 30].”
Poincaré GloVe: Hyperbolic Word Embeddings (2018, methodology): “…graph in a low dimensional space with the aim of improving link prediction either by imposing a partial order in the embedding space (Vendrov et al., 2015; Vilnis et al., 2018; Athiwaratkun & Wilson, 2018), by using hyperbolic geometry (Nickel & Kiela, 2017; 2018), or both (Ganea et al., 2018a).”
Hyperbolic Graph Convolutional Neural Networks (2019, methodology): “We use a similar reasoning to that in Corollary 1.1 in [11].”
Riemannian Adaptive Optimization Methods (2018, background): “…equations (Vander-eycken & Vandewalle, 2010), matrix factorization (Tan et al., 2014), geometric programming (Sra & Hosseini, 2015), dictionary learning (Cherian & Sra, 2017) or hyperbolic taxonomy embedding (Nickel & Kiela, 2017; Ganea et al., 2018a; De Sa et al., 2018; Nickel & Kiela, 2018).”
Representation Tradeoffs for Hyperbolic Embeddings (2018, methodology): “directed acyclic graphs (Ganea et al., 2018b). The authors also propose to optimize on the hyperbolic manifold using its exponential map, as opposed to our approach of finding a closed form for the embedding should it exist (Section 4). An interesting avenue for future work is to compare both optimi”
A Survey on Differential Privacy for Unstructured Data Content (2022, methodology): “Within them, word embedding models including GloVe [73], FastText [7], word2vec [68], and Hyperbolic embedding [39] are selected to transform textual data into high-dimensional vectors; they are then perturbed with noise vectors sampled from probabilistic distributions.”
Context Mover's Distance & Barycenters: Optimal transport of contexts for building representations (2018, methodology): “Apart from embedding entities in the Wasserstein space, other metric spaces like the Hyperbolic space, have recently gained attention for modelling hierarchical structures (Nickel & Kiela, 2017; Ganea et al., 2018).”
Hyperbolic Graph Neural Networks (2019, unknown): “[16] extended Poincaré embeddings to directed graphs using hyperbolic entailment cones.”