Knowledge Graph Completion with Adaptive Sparse Transfer Matrix

By Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao. 2016

467 citations recorded by the source corpus.

We model knowledge graphs for their completion by encoding each entity and relation into a numerical space. All previous work including Trans(E, H, R, and D) ignore the heterogeneity (some relations link many entity pairs and others do not) and the imbalance (the number of head entities and that of tail entities in a relation could be different) of knowledge graphs. In this paper, we propose a novel approach TranSparse to deal with the two issues. In TranSparse, transfer matrices are replaced by adaptive sparse matrices, whose sparse degrees are determined by the number of entities (or entity pairs) linked by relations. In experiments, we design structured and unstructured sparse patterns for transfer matrices and analyze their advantages and disadvantages. We evaluate our approach on triplet classification and link prediction tasks. Experimental results show that TranSparse outperforms Trans(E, H, R, and D) significantly, and achieves state-of-the-art performance.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Minlie Huang
2. Xiaoyan Zhu
3. Liang Yao
4. Chengsheng Mao
5. Yuji Matsumoto
6. Yang Yang
7. Jun Feng
8. Hidekazu Oiwa

Citations in the corpus

• A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, neutral): “TransD [125] kMrhhþ r Mrttk22 TranSparse [126] kM r ður Þhþ r M rðurÞtk 2 l1=2 m-TransH [127] k P r2MðRrÞ arðrÞPnr ðtðrÞÞ þ brk (2); t 2 NMðRrÞ”
• GAKE: Graph Aware Knowledge Embedding (2016, background): “, 2015) X × × TranSparse(Ji et al., 2016) X × × PTransE(Lin et al.”
• Knowledge Representation Learning with Entities, Attributes and Relations (2016, unknown): “, 2015], TranSparse [Ji et al., 2016], KG2E [He et al.”
• A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, unknown): “TranSparse [174] h, t ∈ R r ∈ R ,Mr (θr) ∈ Rk×d −‖Mr (θr) h + r−Mr (θr) t‖21/2 M1r ( θ(1) r ) ,M2r ( θ(2) r ) ∈ Rk×d − ∥∥M1r (θ1 r)h + r−M2r (θ2 r) t∥∥21/2”
• A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, methodology): “Focus then turned to importing hyperspace structures with distance-based translation models for link prediction, such as TransR [243], TransH [244], and TranSparse [245].”
• Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, background): “Here, n andm are the number of entities and relations respectively; d and k the dimensionality of entity and relation embedding space respectively (we usually have d ¼ k); u in TranSparse the average sparseness degree of projection matrices; c in TransG the average number of semantic components per ”
• Knowledge Transfer for Out-of-Knowledge-Base Entities : A Graph Neural Network Approach (2017, methodology): “…r, t) in knowledge base G, a corrupted triplet is generated by substituting a random entity sampled from E(G) for h or t. Specifically, to generate corrupted triplets, we used the “Bernoulli” trick, a technique also used in [Wang et al., 2014b; Lin et al., 2015; Ji et al., 2015; Ji et al., 2016].”
• Knowledge Graph Completion: A Review (2020, background): “[60], which replaces dense matrix in the original model with sparse matrix as mapping matrix, solves the distribution imbalance of entities and relationships in knowledge graph, and reduces the number of parameters in the model at the same time [61].”
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, methodology): “To alleviate heterogeneity and imbalance issues, TranSparse [30] provides an efficient solution by designing adaptive transfer sparser matrices.”
• Analogical Inference for Multi-relational Embeddings (2017, unknown): “…al., 2015b) 92.3 70.2 KG2E (He et al., 2015) 93.2 74.0 STransE (Nguyen et al., 2016) 93.4 79.7 DistMult (Yang et al., 2014) 93.6 82.4 TransSparse (Ji et al., 2016) 93.9 78.3 PTransE-MUL (Lin et al., 2015a) – 77.7 PTransE-RNN (Lin et al., 2015a) – 82.2 PTransE-ADD (Lin et al., 2015a) – 84.6 NLF…”
• KG-BERT: BERT for Knowledge Graph Completion (2019, methodology): “…and its extensions TransH (Wang et al. 2014b), TransD (Ji et al. 2015), TransR (Lin et al. 2015b), TransG (Xiao, Huang, and Zhu 2016), TranSparse (Ji et al. 2016) and PTransE (Lin et al. 2015a), DistMult and its extension DistMult-HRS (Zhang et al. 2018) which only used structural information in…”
• A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network (2017, unknown): “, 2016b) and TranSparse (Ji et al., 2016).”