Representation Learning of Knowledge Graphs with Entity Descriptions

By Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, Maosong Sun. 2016

758 citations recorded by the source corpus.

Representation learning (RL) of knowledge graphs aims to project both entities and relations into a continuous low-dimensional space. Most methods concentrate on learning representations with knowledge triples indicating relations between entities. In fact, in most knowledge graphs there are usually concise descriptions for entities, which cannot be well utilized by existing methods. In this paper, we propose a novel RL method for knowledge graphs taking advantages of entity descriptions. More specifically, we explore two encoders, including continuous bag-of-words and deep convolutional neural models to encode semantics of entity descriptions. We further learn knowledge representations with both triples and descriptions. We evaluate our method on two tasks, including knowledge graph completion and entity classification. Experimental results on real-world datasets show that, our method outperforms other baselines on the two tasks, especially under the zero-shot setting, which indicates that our method is capable of building representations for novel entities according to their descriptions. The source code of this paper can be obtained from https://github.com/xrb92/DKRL.

Subfields: Artificial Intelligence, Management Science and Operations Research, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Representation & Self-Supervised Learning

In Graph Learning

Citations in the corpus

Knowledge Enhanced Contextual Word Representations (2019, methodology): “Other meth-ods combine entity metadata with the graph (Xie et al., 2016), use entity contexts (Chen et al., 2014; Ganea and Hofmann, 2017), or a combination of contexts and the KB (Wang et al., 2014a; Gupta et al., 2017).”
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, neutral): “DKRL [128] khd þ r tdk þ khd þ r tsk þ khs þ r tdk”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, background): “Many works in NLP mainly focus on the zero-shot/semisupervised scenario (Xie et al., 2016; Ma et al., 2016; Carlson et al., 2009), which incorporate ex-”
Pre-trained models for natural language processing: A survey (2020, background): “Early studies [197, 217, 201, 205] focused on learning knowledge graph embeddings and word embedding jointly.”
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, background): “Besides, Logan et al. (2019); Hayashi et al. (2020) utilize relations between entities inside one sentence to help train better generation models, and Xiong et al. (2019) adopt entity replacement knowledge learning for improving entity-related tasks.”
ProjE: Embedding Projection for Knowledge Graph Completion (2016, positive): “Another curious ﬁnding is that some of the existing models are not self-contained models , i.e. , they require pre-trained KG embeddings (RTransE, CVSM), pre-selected paths (PTransE, RTransE), or pre-computed content embeddings of each node (DKRL [36]) before their model training can even begin.”
Semantic Proximity Search on Heterogeneous Graph by Proximity Embedding (2017, methodology): “More recent methods use neural networks to learn node embedding by either shallow architectures (Yang, Cohen, and Salakhutdinov 2016; Xie et al. 2016; Tang et al. 2015) or deep architectures (Niepert, Ahmed, and Kutzkov 2016; Wang, Cui, and Zhu 2016; Chang et al. 2015).”
Incorporating Knowledge Graph Embeddings into Topic Modeling (2017, positive): “Knowledge graph embedding aims at embedding entities and relationships of knowledge graphs in vector spaces (Bordes et al. 2011; 2013; Wang et al. 2014b; 2014a; Guo et al. 2015; Xie et al. 2016).”
From Node Embedding To Community Embedding (2016, methodology): “Recent methods typically rely on neural networks to learn the representation for each node, with either shallow architectures [Xie et al., 2016; Tang et al., 2015; Grover and Leskovec, 2016] or deep architectures [Niepert et al., 2016; Wang et al., 2016; Chang et al., 2015].”
Learning Community Embedding with Community Detection and Node Embedding on Graphs (2017, background): “…extracting the leading eigenvectors of a graph affinity matrices. embed. detection complexity DeepWalk [20] • More recent methods start to exploit neural networks to learn the representation for each node, with either shallow architectures [12, 24, 33] or deep architectures [1, 8, 18, 29].”
Open-World Knowledge Graph Completion (2017, methodology): “DKRL is a two-layer CNN model that generates entity embeddings with entity description (Xie et al. 2016).”
CoLAKE: Contextualized Language and Knowledge Embedding (2020, methodology): “For inductive setting, we take DKRL (Xie et al., 2016) as our baseline.”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, background): “, entity types [25], [26], relation paths [27], [28], [29], textual descriptions [30], [31], [32], [33], and even logical rules [23], [34], [35], to learn more pre-”
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, background): “Then, DKRL [249] takes advantage of entity descriptions in KGs with language neural networks and gets more expressive semantics for reasoning.”
SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions (2016, methodology): “DKRL (Xie et al. 2016) extends the translationbased embedding methods from the triple-specific one to the “Text-Aware” model.”
Multi-Modal Knowledge Graph Construction and Application: A Survey (2022, background): “Link prediction in MMKG [134], [136], [141], [142] aims to complete a triple (h, r, t) when one of the entities in h, r, t”
Representation Learning of Knowledge Graphs with Hierarchical Types (2016, background): “DKRL [Xie et al., 2016] proposes description-based representations for entities constructed from entity descriptions with CBOW or CNN, which is capable of modeling entities in zero-shot scenario.”
Knowledge Graph Embedding Based Question Answering (2019, background): “Xie et al. [42] and Fan et al. [17] explored the semantic information in entity descriptions to advance KG embedding.”