SimplE Embedding for Link Prediction in Knowledge Graphs
By Seyed Mehran Kazemi, David Poole. Neural Information Processing Systems, 2018
857 citations recorded by the source corpus.
Knowledge graphs contain knowledge about the world and provide a structured representation of this knowledge. Current knowledge graphs contain only a small subset of what is true in the world. Link prediction approaches aim at predicting new links for a knowledge graph given the existing links among the entities. Tensor factorization approaches have proved promising for such link prediction problems. Proposed in 1927, Canonical Polyadic (CP) decomposition is among the first tensor factorization approaches. CP generally performs poorly for link prediction as it learns two independent embedding vectors for each entity, whereas they are really tied. We present a simple enhancement of CP (which we call SimplE) to allow the two embeddings of each entity to be learned dependently. The complexity of SimplE grows linearly with the size of embeddings. The embeddings learned through SimplE are interpretable, and certain types of background knowledge can be incorporated into these embeddings through weight tying. We prove SimplE is fully expressive and derive a bound on the size of its embeddings for full expressivity. We show empirically that, despite its simplicity, SimplE outperforms several state-of-the-art tensor factorization techniques. SimplE's code is available on GitHub at this https URL.
Subfields: Artificial Intelligence, Computational Mathematics
View Paper: https://www.semanticscholar.org/paper/70af3ee98c53441d9090119f7b76efb1b6d03edd
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Partha P. Talukdar
- Apoorv Saxena
- Yi Tay
- Zequn Sun
- Wei Hu
- Timothy M. Hospedales
- Yoshua Bengio
- Mohit Bansal
In Computational Mathematics
- Partha P. Talukdar
- Apoorv Saxena
- Ivana Balazevic
- Carl Allen
- Timothy M. Hospedales
- Nicolas Usunier
- Aditay Tripathi
- Timothée Lacroix
Citations in the corpus
- A benchmarking study of embedding-based entity alignment for knowledge graphs (2020, positive): “Existing KG embedding models can be generally divided into three categories: (i) translational models , e.g., TransE [5], TransH [82], TransR [49] and TransD [33]; (ii) semantic matching models , e.g., DistMult [86], ComplEx [76], HolE [54], SimplE [36], RotatE [71] and TuckER [3]; and (iii) deep mo”
- Time2Vec: Learning a Vector Representation of Time (2019, methodology): “Vector embedding has been previously successfully used for other domains such as text [40, 48], (knowledge) graphs [22, 46, 25], and positions [57, 17].”
- Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey (2019, methodology): “For KG embedding, most existing approaches rely on shallow encoders (see e.g., [171, 241, 218, 17, 168, 115, 61]) with a few exceptions.”
- Tensor Decompositions for temporal knowledge base completion (2020, methodology): “This method has lead to many variations (Ji et al., 2015; Nguyen et al., 2016; Wang et al., 2014), but is limited in the relation systems it can model (Kazemi & Poole, 2018) and does not lead to state of the art performances on current benchmarks.”
- TuckER: Tensor Factorization for Knowledge Graph Completion (2019, background): “A large number of approaches to link prediction so far have been linear, based on various meth-ods of factorizing the third-order binary tensor (Nickel et al., 2011; Yang et al., 2015; Trouillon et al., 2016; Kazemi and Poole, 2018).”
- Representation Learning for Dynamic Graphs: A Survey (2019, methodology): “Realizing the information may not flow well between the two vectors of a node, SimplE adds another vector to the relation embeddings as EMB(r) = (zr, zr) where zr models the behaviour of the inverse of the relation.”
- Knowledge Graph Embedding for Link Prediction (2020, methodology): “Examples of models that have been demonstrated to be fully expressive are SimplE [27], TuckER [6], ComplEx [61] and HolE [60].”
- Quaternion Knowledge Graph Embeddings (2019, background): “SimplE [Kazemi and Poole, 2018] is also a simple yet effective bilinear approach for knowledge graph embedding.”
- Knowledge Graphs (2020, neutral): “SimplE [68] proposes to compute a standard CP decomposition, averaging terms across X, Y, Z to compute the final plausibility scores.”
- Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings (2020, methodology): “SimplE (Kazemi and Poole, 2018) and TuckER (Balaˇzevi´c et al., 2019) are based on Canonical Polyadic (CP) decomposition (Hitchcock, 1927) and Tucker decomposition (Tucker, 1966) respectively.”
- Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction (2021, background): “Later works improve embedding methods with new score functions [58, 13, 31, 52, 54, 76] that capture common semantic patterns of the relations, or search the score function in a general design space [75].”
- Neural, symbolic and neural-symbolic reasoning on knowledge graphs (2021, methodology): “P Q (Bordes et al., 2013) force h þ r to be close to t TransH (Wang et al., 2014) project entities into the relation-speci fi c hyperplane TransR (Lin et al., 2015) project entities and relations in separate spaces TransD (Ji et al., 2015) determine the project matrices by both entities and relations”
- Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation (2019, methodology): “2016), SimplE (Kazemi and Poole 2018) and RotatE (Sun et al.”
- TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery (2022, methodology): “TransE (Bordes et al., 2013) DistMult (Yang et al., 2014) ComplEx (Trouillon et al., 2016) NeuralLP (Yang et al., 2017) SimplE (Kazemi and Poole, 2018) RotatE (Sun et al., 2019b) KBGAT (Nathani et al., 2019)”
- Deep Bidirectional Language-Knowledge Graph Pretraining (2022, neutral): “(cid:22)(cid:107)(cid:104)(cid:81)(cid:93)(cid:91)(cid:3)(cid:32)(cid:60)(cid:115)(cid:73)(cid:103) (cid:164)(cid:164)(cid:164) (cid:12)(cid:17)(cid:23) (cid:12)(cid:73)(cid:79) (cid:164)(cid:164)(cid:164) (cid”