Neural Embeddings of Graphs in Hyperbolic Space
By Benjamin Paul Chamberlain, James R. Clough, Marc Peter Deisenroth. 2017
188 citations recorded by the source corpus.
Neural embeddings have been used with great success in Natural Language Processing (NLP). They provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks. The success of neural embeddings has prompted significant amounts of research into applications in domains other than language. One such domain is graph-structured data, where embeddings of vertices can be learned that encapsulate vertex similarity and improve performance on tasks including edge prediction and vertex labelling. For both NLP and graph based tasks, embeddings have been learned in high-dimensional Euclidean spaces. However, recent work has shown that the appropriate isometric space for embedding complex networks is not the flat Euclidean space, but negatively curved, hyperbolic space. We present a new concept that exploits these recent insights and propose learning neural embeddings of graphs in hyperbolic space. We provide experimental evidence that embedding graphs in their natural geometry significantly improves performance on downstream tasks for several real-world public datasets.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Jure Leskovec
- Sergey Levine
- Quoc V. Le
- Mohit Bansal
- William L. Hamilton
- Percy Liang
- Ruslan Salakhutdinov
In Statistical and Nonlinear Physics
- William L. Hamilton
- Bryan Perozzi
- Rex Ying
- Jure Leskovec
- Christopher R'e
- Caglar Gulcehre
- Pascal Poupart
- Rishab Goel
Citations in the corpus
- Representation Learning on Graphs: Methods and Applications (2017, methodology): “…algorithm to learn embeddings using random walks that “skip” or “hop” over multiple nodes at each step, resulting in a proximity measure similar to GraRep [9], while Chamberlan et al. [11] modify the inner-product decoder of node2vec to use a hyperbolic, rather than Euclidean, distance measure.”
- Representation Learning for Dynamic Graphs: A Survey (2019, unknown): “Instead of computing the distance between zv and zu in the Euclidean space, the distance can be computed in other spaces such as the hyperbolic space (see, e.g., Chamberlain et al. (2017)).”
- Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey (2019, background): “…value of the difference of the two vectors: abs ( z v − z u ) , • The element-wise squared value of the difference of the two vectors: Instead of computing the distance between z v and z u in the Euclidean space, the distance can be computed in other spaces such as the hyperbolic space [32].”
- Hyperbolic Attention Networks (2018, background): “Following this exciting line of research, recently the machine learning community has gained interest in learning non-Euclidean embeddings directly from data [28, 8, 33, 29, 38, 6].”
- Machine Learning on Graphs: A Model and Comprehensive Taxonomy (2020, methodology): “[21] extend Poincaré embeddings to incorporate skip-gram losses using hyperbolic inner products.”
- Representation Tradeoffs for Hyperbolic Embeddings (2018, methodology): “The use of dH(x, y) is a minor instability in Nickel & Kiela (2017); Chamberlain et al. (2017)’s formulation, necessitating guarding against NANs.”