Learning Entity and Relation Embeddings for Knowledge Graph Completion
By Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu. 2015
3,991 citations recorded by the source corpus.
Knowledge graph completion aims to perform link prediction between entities. In this paper, we consider the approach of knowledge graph embeddings. Recently, models such as TransE and TransH build entity and relation embeddings by regarding a relation as translation from head entity to tail entity. We note that these models simply put both entities and relations within the same semantic space. In fact, an entity may have multiple aspects and various relations may focus on different aspects of entities, which makes a common space insufficient for modeling. In this paper, we propose TransR to build entity and relation embeddings in separate entity space and relation spaces. Afterwards, we learn embeddings by first projecting entities from entity space to corresponding relation space and then building translations between projected entities. In experiments, we evaluate our models on three tasks including link prediction, triple classification and relational fact extraction. Experimental results show significant and consistent improvements compared to state-of-the-art baselines including TransE and TransH.
Subfields: Artificial Intelligence, Management Science and Operations Research, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Zhiyuan Liu
- Percy Liang
- Luke S. Zettlemoyer
- Maosong Sun
- Noah A. Smith
- Yejin Choi
In Management Science and Operations Research
In Graph Learning
- Maosong Sun
- Zhiyuan Liu
- Tim Weninger
- Baoxu Shi
- Minlie Huang
- Xiaoyan Zhu
- William Yang Wang
- William L. Hamilton
Citations in the corpus
- Word-Entity Duet Representations for Document Ranking (2017, methodology): “easy to imagine further developments in various places. For example, the recent approaches in neural ranking with word embeddings can be incorporated [9]; be−er knowledge graph embeddings can be used [13]; be−er entity search methods can be applied when extracting word to entity features [3]; the a−”
- Knowledge Enhanced Contextual Word Representations (2019, methodology): “These methods broadly fall into two categories: translational distance models (Bordes et al., 2013; Wang et al., 2014b; Lin et al., 2015; Xiao et al., 2016) which use a distance-based scoring function, and linear models (Nickel et al., 2011; Yang et al., 2014; Trouillon et al., 2016; Dettmers et…”
- Knowledge Graph Convolutional Networks for Recommender Systems (2019, background): “, TransE [1] and TransR [12] assume head + relation = tail ), which are more suitable for in-graph applications such as KG completion and link prediction rather than recommendation.”
- RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems (2018, methodology): “tor-movie"). •LibFM[20] is a widely used feature-based factorization model in CTR scenarios. We concatenate user ID, item ID, and the corresponding averaged entity embeddings learned from TransR [13] as input for LibFM. •DeepWide [6] is a general deep model for recommendation combining a (wide)”
- DKN: Deep Knowledge-Aware Network for News Recommendation (2018, methodology): “In addition to translation-based methods [4, 18, 26, 48] used in DKN, researchers have also proposed many other models such as Structured Embedding [5], Latent Factor Model [17], Neural Tensor Network [37] and GraphGAN [42].”
- A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, background): “Triplet classification [14], [15], [38], [51], [52], [53], [61], [142] is a specific application for knowledge graph.”
- KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, background): “The intuition is that semantics of an entity may have multiple aspects, and different relations focus on different ones (Lin et al., 2015).”
- Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge (2020, background): “More recently, the past decade has seen huge amount of work on knowledge base embeddings (Bordes et al., 2013; Lin et al., 2015; Trouillon et al., 2017; Dettmers et al., 2018) which enable generalization through similarities between”
- GRAM: Graph-based Attention Model for Healthcare Representation Learning (2016, background): “They aim to project entities and relations (Bordes et al., 2013; Socher et al., 2013; Wang et al., 2014; Lin et al., 2015) to the latent space based on the triples or additional information such as hierarchy of entities (Xie et al., 2016).”
- Knowledge Graph Embedding via Dynamic Mapping Matrix (2015, methodology): “As we construct negative triplets for FB15k by ourselves, we use the codes of TransE, TransH and TransR/CTransR provied by (Lin et al. 2015) to evaluate the datasets instead of reporting the re-sults of (Wang et al.2014; Lin et al. 2015) directly.”
- Holographic Embeddings of Knowledge Graphs (2015, methodology): “(2014) and Lin et al. (2015) proposed TRANS H and TRANS R respectively, to improve the performance of T RANS E on 1-to-N, N-to-1, and N-to-N relations.”
- Prototypical Representation Learning for Relation Extraction (2021, background): “Although many works are proposed to learn representations for relations from well-structured knowledge (Bordes et al., 2013; Lin et al., 2015; Ji et al., 2015), when we extend the learning source to be unstructured distantly-labeled text (Mintz et al., 2009), this task becomes particularly…”
- Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “, deterministic points in the vector space [13], [14], [15], [16], [19].”
- SSE: Semantically Smooth Embedding for Knowledge Graphs (2017, background): “Both are formulated as geometrically based regularization terms to constrain the embedding task.”
- A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network (2017, methodology): “Previous work shows that TransE obtains very competitive results (Lin et al., 2015a; Nickel et al., 2016b; Trouillon et al., 2016; Nguyen et al., 2016a).”
- KG-BERT: BERT for Knowledge Graph Completion (2019, methodology): “…multiple state-of-the-art KG embedding methods as follows: TransE and its extensions TransH (Wang et al. 2014b), TransD (Ji et al. 2015), TransR (Lin et al. 2015b), TransG (Xiao, Huang, and Zhu 2016), TranSparse (Ji et al. 2016) and PTransE (Lin et al. 2015a), DistMult and its extension…”
- A Survey on Accuracy-Oriented Neural Recommendation: From Collaborative Filtering to Information-Rich Recommendation (2021, methodology): “CKE [132] makes use of Knowledge Graph Embedding (KGE) techniques, especially TransR [197], to generate additional representations of items, and then inte-”
- HittER: Hierarchical Transformers for Knowledge Graph Embeddings (2020, background): “Early work uses translational distance-based scoring functions defined on top of entity and relation embeddings (Bordes et al., 2013; Wang et al., 2014; Lin et al., 2015; Ji et al., 2015).”
- A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, background): “2020 GraIL [7] Inductive GNN 2015 TransR [19] Transductive Translational”
- A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, methodology): “Focus then turned to importing hyperspace structures with distance-based translation models for link prediction, such as TransR [243], TransH [244], and TranSparse [245].”