Holographic Embeddings of Knowledge Graphs
By Maximilian Nickel, Lorenzo Rosasco, Tomaso A. Poggio. 2015
1,271 citations recorded by the source corpus.
Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator, HolE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. Experimentally, we show that holographic embeddings are able to outperform state-of-the-art methods for link prediction on knowledge graphs and relational learning benchmark datasets.
Subfields: Artificial Intelligence, Management Science and Operations Research, Representation & Self-Supervised Learning, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Zhiyuan Liu
- Percy Liang
- Luke S. Zettlemoyer
- Yoshua Bengio
- Maosong Sun
- Sebastian Riedel
In Management Science and Operations Research
In Representation & Self-Supervised Learning
- Bilal Abu-Salih
- Partha P. Talukdar
- Seyed Mehran Kazemi
- Xiang Wang
- Xing Xie
- Xiangnan He
- Tat-seng Chua
- Jure Leskovec
In Knowledge Graphs & Representation
Citations in the corpus
- RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems (2018, methodology): “the observed knowledge graph Ggiven Θ. Recently, researchers have proposed a great many knowledge graph embedding methods, including translational distance models [3, 14] and semantic matching models [15,19](WewillcontinuethediscussiononKGEmethodsinSection 3.6.3). In RippleNet, we use a three-way te”
- Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning (2019, methodology): “2014), and HOLE (Nickel et al. 2016) for the creation of embeddings from KGs.”
- A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, background): “HOLE [132] r ðh ? tÞ, where ? is circular correlation”
- Modeling Relational Data with Graph Convolutional Networks (2017, methodology): “Numerous alternative factorizations have been proposed and studied in the context of SRL, including both (bi-)linear and non-linear ones (e.g., (Bordes et al. 2013; Socher et al. 2013; Chang et al. 2014; Nickel, Rosasco, and Poggio 2015; Trouillon et al. 2016)).”
- Matching the Blanks: Distributional Similarity for Relation Learning (2019, positive): “This is inspired by recent work in knowledge base embedding (Bordes et al., 2013; Nickel et al., 2016).”
- Complex Embeddings for Simple Link Prediction (2016, methodology): “However, due to time constraints and the complexity of an efficient implementation of HolE, we record the original results for HolE as reported in Nickel et al. (2016b).”
- End-to-end Differentiable Proving (2017, positive): “We follow [36] and split countries randomly into a training set of 204 countries (train), a development set of 20 countries (dev), and a test set of 20 countries (test), such that every dev and test country has at least one neighbor in the training set.”
- Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (2019, background): “Asymmetry has also demonstrated utility in domains such as relational learning (Trouillon et al., 2016; Nickel et al., 2016) and question answering (Tay et al., 2018).”
- PyTorch-BigGraph: A Large-scale Graph Embedding System (2019, methodology): “Raw Filtered Hit@10 RESCAL (Nickel et al., 2011) 0.189 0.354 0.587 TransE (Bordes et al., 2013) 0.222 0.463 0.749 HolE (Nickel et al., 2016b) 0.232 0.524 0.739 ComplEx (Trouillon et al., 2016) 0.242 0.692 0.840 R-GCN+ (Schlichtkrull et al., 2018) 0.262 0.696 0.842 StarSpace (Wu et al.,”
- Domain-specific Knowledge Graphs: A survey (2020, methodology): “Example of KGE models include, but not limited to; Translating Embedding (TransE) [35], DistMult [36], Complex Embeddings (ComplEx) [37], Holographic Embeddings(HolE) [38], Convolutional 2D KG Embeddings (ConvE) [39], Convolution-based model (ConvKB) [40], etc.”
- A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, background): “…information, specific strategies are designed for specific information, like circular correlations, analogical structural information, etc. Specifically, HolE [47] models circular correlation with compressed representations, and HolEx [48] further generates the expanded holographic embeddings.”
- Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “TABLE 2 Summary of Semantic Matching Models (See Section 3.2 for Details) Method Ent. embedding Rel. embedding Scoring function frðh; tÞ Constraints/Regularization RESCAL [13] h; t 2 Rd Mr 2 Rd d h>Mrt khk2 1; ktk2 1; kMrkF 1 Mr ¼ P i p i ruiv > i (required in [17]) TATEC [64] h; t 2 Rd r 2 Rd;Mr 2 ”
- OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs (2019, methodology): “edge prediction), which include neural models trained to discern positive and negative triples (Bordes et al., 2013; Wang et al., 2014; Nickel et al., 2016; Dettmers et al., 2018), or algorithms with discrete KG operations on structured data (Lao et al.”
- Iterative Entity Alignment via Joint Knowledge Embeddings (2017, methodology): “There are existing some other methods to learn knowledge representations: RESCAL [Nickel et al., 2011; 2012] and HOLE [Nickel et al., 2016] are based on tensor factorization with relations considered to be matrices.”
- Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences (2019, methodology): “To simplify the learning of relation matrices, DistMult [43] restricts them to be diagonal, HolE [26] defines a circular correlation [32] to compress relation matrices as vectors, and ComplEx [38] introduces complex-value for asymmetric relations.”
- SimplE Embedding for Link Prediction in Knowledge Graphs (2018, methodology): “We report the results of TransR and NTN from [27], and ER-MLP from [32] for further comparison.”