A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network

By Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Q. Phung. North American Chapter of the Association for Computational Linguistics, 2017

829 citations recorded by the source corpus.

In this paper, we propose a novel embedding model, named ConvKB, for knowledge base completion. Our model ConvKB advances state-of-the-art models by employing a convolutional neural network, so that it can capture global relationships and transitional characteristics between entities and relations in knowledge bases. In ConvKB, each triple (head entity, relation, tail entity) is represented as a 3-column matrix where each column vector represents a triple element. This 3-column matrix is then fed to a convolution layer where multiple filters are operated on the matrix to generate different feature maps. These feature maps are then concatenated into a single feature vector representing the input triple. The feature vector is multiplied with a weight vector via a dot product to return a score. This score is then used to predict whether the triple is valid or not. Experiments show that ConvKB achieves better link prediction performance than previous state-of-the-art embedding models on two benchmark datasets WN18RR and FB15k-237.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/41cbe3cdfee7539f8ff20144e6bd028ef762e00c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Knowledge Graphs: Opportunities and Challenges (2023, methodology): “ConvKB (Nguyen et al. 2017) utilizes a convolutional neural network (CNN) to conduct knowledge graph embeddings.”
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning (2020, neutral): “TransE (Bordes et al., 2013) - 75.9 NTN (Socher et al., 2013) - 86.2 DistMult (Yang et al., 2015) - 87.1 DistMult-HRS (Yang et al., 2015) - 88.9 DOLORES (Wang et al., 2018) - 87.5 ConvKB (Nguyen et al., 2018) - 87.6 AATE (An et al., 2018) - 88.0 KG-BERT (BERT-base) (Yao et al., 2019) - 93.5 GLM (BER”
Knowledge Graph Embedding for Link Prediction (2020, methodology): “ConvKB [42] models entities and relations as same-sized one-dimensional embeddings.”
Revisiting Evaluation of Knowledge Base Completion Models (2020, background): “…prediction task, include tensor multiplication [Nickel et al., 2011, Socher et al., 2013, Yang et al., 2015, Balaževi´c et al., 2019], algebraic operations [Bordes et al., 2011, 2013b, Dasgupta et al., 2018, Sun et al., 2019a], and complex neural models [Dettmers et al., 2018, Nguyen et al., 2018].”
InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions (2019, methodology): “As noted in (Shang et al. 2019), although ConvKB was claimed to be superior to ConvE, its performance is not consistent across different datasets and metrics.”
Adaptive Convolution for Multi-Relational Learning (2019, background): “, 2018), and ConvKB (Nguyen et al., 2018).”
End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion (2018, positive): “The most recent KG embedding models are ConvE (Dettmers et al. 2017) and ConvKB (Nguyen et al. 2017).”
Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion (2020, background): “Recently, some semantic matching graph embedding approaches for KGC use deterministic strategy [22, 23] to learn the representation of entities and relations.”
Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020, background): “ConvE and ConvKB (Dettmers et al., 2018; Nguyen et al., 2018) leverage the expressiveness of convolutional neural networks to learn entity embeddings and relation embeddings.”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “ConMask [84] proposes relationship-dependent content masking over the entity description to select relevant snippets of given relations, and CNN-based target fusion to complete the knowledge graph with unseen entities.”
KG-BERT: BERT for Knowledge Graph Completion (2019, background): “(cid:104) (cid:105) Recently, convolutional neural networks also show promising results for knowledge graph completion (Dettmers et al. 2018; Nguyen et al. 2018a; Schlichtkrull et al. 2018).”
Neural, symbolic and neural-symbolic reasoning on knowledge graphs (2021, methodology): “…2018) use the head and relation embeddings as the input of CNN ConvR (Jiang et al., 2019) extend global ﬁ lters in ConvE to relation-speci ﬁ c ﬁ lters ConvKB (Nguyen et al., 2018) use the head, relation and tail together as the input of CNN CapsE (Vu et al., 2019) use the capsule network as…”
A benchmarking study of embedding-based entity alignment for knowledge graphs (2020, neutral): “, ProjE [75], ConvE [14], ConvKB [61], R-GCN [72] and KBGAN [7].”
Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction (2019, methodology): “Results of TransE and RotatE are taken from Nguyen et al. (2018) and Sun et al. (2019), respectively.”
Quaternion Knowledge Graph Embeddings (2019, background): “…of all quaternions and remove the normalization step, the scoring function becomes φ ( h, r, t ) = (cid:104) a h , a r , a t (cid:105) , degrading to DistMult in this case. et al., 2017]; [ (cid:5) ]: Results are taken from [Dettmers et al., 2018]; [ ∗ ]: Results are taken from [Sun et al., 2019].”
Domain-specific Knowledge Graphs: A survey (2020, methodology): “Example of KGE models include, but not limited to; Translating Embedding (TransE) [35], DistMult [36], Complex Embeddings (ComplEx) [37], Holographic Embeddings(HolE) [38], Convolutional 2D KG Embeddings (ConvE) [39], Convolution-based model (ConvKB) [40], etc.”
Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs (2019, methodology): “These methods can be broadly classiﬁed as: (i) compositional, (ii) translational, (iii) CNN based, and (iv) graph based models.”