Poincaré GloVe: Hyperbolic Word Embeddings
By A. Tifrea, Gary Bécigneul, Octavian-Eugen Ganea. International Conference on Learning Representations, 2018
343 citations recorded by the source corpus.
Words are not created equal. In fact, they form an aristocratic graph with a latent hierarchical structure that the next generation of unsupervised learned word embeddings should reveal. In this paper, justified by the notion of delta-hyperbolicity or tree-likeliness of a space, we propose to embed words in a Cartesian product of hyperbolic spaces which we theoretically connect to the Gaussian word embeddings and their Fisher geometry. This connection allows us to introduce a novel principled hypernymy score for word embeddings. Moreover, we adapt the well-known Glove algorithm to learn unsupervised word embeddings in this type of Riemannian manifolds. We further explain how to solve the analogy task using the Riemannian parallel transport that generalizes vector arithmetics to this new type of geometry. Empirically, based on extensive experiments, we prove that our embeddings, trained unsupervised, are the first to simultaneously outperform strong and popular baselines on the tasks of similarity, analogy and hypernymy detection. In particular, for word hypernymy, we obtain new state-of-the-art on fully unsupervised WBLESS classification accuracy.
Subfields: Artificial Intelligence, Information Systems
View Paper: https://www.semanticscholar.org/paper/e8fa823c17aeb8d08fe9aa5fc2bc0eaacb9edcdf
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jure Leskovec
- Christopher R'e
- Rex Ying
- William L. Hamilton
- Ines Chami
- Martin Jaggi
- Bryan Perozzi
- Timothy M. Hospedales
In Information Systems
- Jure Leskovec
- Rex Ying
- Bryan Perozzi
- Timothy M. Hospedales
- Kevin P. Murphy
- Christopher R'e
- Martin Jaggi
- Aymeric Dieuleveut
Citations in the corpus
- Multi-relational Poincaré Graph Embeddings (2019, methodology): “However, a clear correspondence to the Euclidean inner product does not exist in hyperbolic space [30].”
- Context Mover's Distance & Barycenters: Optimal transport of contexts for building representations (2018, methodology): “8, 34 [80] Alexandru Tifrea, Gary Bécigneul, and Octavian-Eugen Ganea.”
- Hyperbolic Graph Convolutional Neural Networks (2019, methodology): “Shallow embedding methods have also been developed in hyperbolic geometry [28, 27] for reconstructing trees [33] and graphs [21, 5], or embedding text [37].”
- Machine Learning on Graphs: A Model and Comprehensive Taxonomy (2020, background): “For instance, hyperbolic embeddings can represent hierarchical data with much smaller distortion than Euclidean embeddings [131] and have led to state-of-the-art results in many modern applications such as link prediction in knowledge graphs [10, 33] and linguistics tasks [92, 142].”
- Hyperspherical Prototype Networks (2019, neutral): “Several works have investigated prior and semantic knowledge in hyperbolic spaces [8, 32, 40].”