Complex Embeddings for Simple Link Prediction
By Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard. 2016
3,691 citations recorded by the source corpus.
In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to solve this problem through latent factorization. However, here we make use of complex valued embeddings. The composition of complex embeddings can handle a large variety of binary relations, among them symmetric and antisymmetric relations. Compared to state-of-the-art models such as Neural Tensor Network and Holographic Embeddings, our approach based on complex embeddings is arguably simpler, as it only uses the Hermitian dot product, the complex counterpart of the standard dot product between real vectors. Our approach is scalable to large datasets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.
Subfields: Artificial Intelligence, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Yoshua Bengio
- Percy Liang
- Luke S. Zettlemoyer
- Quoc V. Le
- Graham Neubig
- Caiming Xiong
In Statistical and Nonlinear Physics
- Jure Leskovec
- William L. Hamilton
- Philip S. Yu
- Shirui Pan
- Marinka Zitnik
- Jian Tang
- Yuxiao Dong
- Joseph Fisher
In Representation & Self-Supervised Learning
- William L. Hamilton
- Jure Leskovec
- Yoshua Bengio
- Xipeng Qiu
- Philip S. Yu
- Zhiyuan Liu
- Sebastian Ruder
- Marinka Zitnik
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Jian Tang
- Marinka Zitnik
- Matthias Fey
- Stephan Günnemann
- Michael M. Bronstein
- Xiangnan He
Citations in the corpus
- Knowledge Enhanced Contextual Word Representations (2019, methodology): “…distance models (Bordes et al., 2013; Wang et al., 2014b; Lin et al., 2015; Xiao et al., 2016) which use a distance-based scoring function, and linear models (Nickel et al., 2011; Yang et al., 2014; Trouillon et al., 2016; Dettmers et al., 2018) which use a similarity-based scoring function.”
- Integrating Learning and Reasoning with Deep Logic Models (2019, background): “Neural-symbolic approaches have been proved to be very powerful to perform approximated logical reasoning [29].”
- Compositional Fairness Constraints for Graph Embeddings (2019, background): “, [5, 20, 22]) are also amenable to our framework.”
- Modeling polypharmacy side effects with graph convolutional networks (2018, methodology): “, 2013; Trouillon et al. , 2016), we estimate the model through negative sampling.”
- Modeling Relational Data with Graph Convolutional Networks (2017, methodology): “A simple baseline LinkFeat employing a linear classifier on top of sparse feature vectors of observed training relations was shown to outperform existing systems by a large margin.”
- End-to-end Differentiable Proving (2017, methodology): “To speed up learning subsymbolic representations, we train NTPs jointly with ComplEx [7] (Appendix B).”
- Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (2019, background): “Asymmetry has also demonstrated utility in domains such as relational learning (Trouillon et al., 2016; Nickel et al., 2016) and question answering (Tay et al.”
- Domain-specific Knowledge Graphs: A survey (2020, methodology): “Example of KGE models include, but not limited to; Translating Embedding (TransE) [35], DistMult [36], Complex Embeddings (ComplEx) [37], Holographic Embeddings(HolE) [38], Convolutional 2D KG Embeddings (ConvE) [39], Convolution-based model (ConvKB) [40], etc.”
- Knowledge Graphs (2020, neutral): “ComplEx [525], on the other hand, uses a complex vector (i.”
- Representation Learning for Visual-Relational Knowledge Graphs (2017, background): “…d r i b u The training objective is often based on the logistic loss, which has been shown to be superior for most of the composition functions [47], where T pos and T neg are the set of positive and negative training triples, respectively, Θ are the parameters trained during learning and λ is…”
- Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks (2020, background): “Another class of models are the semantic matching models that measures plausibility of facts using a triangular norm (Yang et al. 2015; Trouillon et al. 2016; Sun et al. 2019).”
- Canonical Tensor Decomposition for Knowledge Base Completion (2018, methodology): “To assess whether the results we obtain are specific to CP, we also carry on the same experiments with a state-of-the-art model, ComplEx (Trouillon et al., 2016).”
- Questions for Flat-Minima Optimization of Modern Neural Networks (2022, methodology): “, 2019), ComplEx (Trouillon et al., 2016), and CP (Lacroix et al.”
- Deep learning for noise-tolerant RDFS reasoning (2019, methodology): “This category includes RESCAL [59], DistMul [60] where Mr is simplified to a diagonal matrix, ComplEx [61] which extends DistMul by using complex valued embedding in order to support asymmetric relations.”
- A Survey of Fake News (2020, background): “…NTN [Socher et al. 2013], DistMult [Yang et al. 2014], TransE [Bordes et al. 2013], TransH [Wang et al. 2014], TransR [Lin et al. 2015], Com-plEx [Trouillon et al. 2016], SimplE [Kazemi and Poole 2018], and ConMask [Shi and Weninger 2018]); (ii) graph feature models , which assume that the…”