Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

By Deepak Nathani, Jatin Chauhan, Charu Sharma, Manohar Kaul. Annual Meeting of the Association for Computational Linguistics, 2019

581 citations recorded by the source corpus.

The recent proliferation of knowledge graphs (KGs) coupled with incomplete or partial information, in the form of missing relations (links) between entities, has fueled a lot of research on knowledge base completion (also known as relation prediction). Several recent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction. However, we observe that these KG embeddings treat triples independently and thus fail to cover the complex and hidden information that is inherently implicit in the local neighborhood surrounding a triple. To this effect, our paper proposes a novel attention-based feature embedding that captures both entity and relation features in any given entity’s neighborhood. Additionally, we also encapsulate relation clusters and multi-hop relations in our model. Our empirical study offers insights into the efficacy of our attention-based model and we show marked performance gains in comparison to state-of-the-art methods on all datasets.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/fddd3dab90c243ab7fc038bc6449ef62c0e06037

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Philip S. Yu
3. Erik Cambria
4. Percy Liang
5. Xiaodong Liu
6. Uri Alon
7. Shirui Pan
8. Christopher R'e

Citations in the corpus

• A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, background): “Neural network-based encoding models start from distributed representation of entities and relations, and some utilizes complex neural structures such as tensor networks [18], graph convolution networks [44], [60], [62], recurrent networks [45]”
• Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020, methodology): “More recently, the KBGAT (Nathani et al., 2019) and A2N (Bansal et al., 2019) models use graph attention networks for knowledge graph embed-dings.”
• Representation Learning for Dynamic Graphs: A Survey (2019, background): “Wang et al. (2019) and Nathani et al. (2019) propose attention-based variants of Equation (17).”
• HittER: Hierarchical Transformers for Knowledge Graph Embeddings (2020, result): “Nathani et al. (2019) also propose an attention-based feature embedding to capture multihop neighbor information, but unfortunately, their reported results have been proven to be unreliable in a recent re-evaluation (Sun et al., 2020).”
• Neural, symbolic and neural-symbolic reasoning on knowledge graphs (2021, methodology): “P Q (Bordes et al., 2013) force h þ r to be close to t TransH (Wang et al., 2014) project entities into the relation-speci fi c hyperplane TransR (Lin et al., 2015) project entities and relations in separate spaces TransD (Ji et al., 2015) determine the project matrices by both entities and relations”
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal (2022, methodology): “In particular, KBGAT [59] leverages attention-based feature embeddings for better reasoning performance.”
• How Attentive are Graph Attention Networks? (2021, background): “…domains (Wang et al., 2019a; Qiu et al., 2018; Yang et al., 2020; Wang et al., 2019c; Huang and Carley, 2019; Ma et al., 2020; Kosaraju et al., 2019; Nathani et al., 2019; Wu et al., 2020; Zhang et al., 2020) and has become the default implementation of “graph attention network” in all popular…”
• Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey (2023, unknown): “A structure-aware convolutional network (SACN) takes the benefit of GCN and ConvE[208] together, where GCN as the encoder utilizes knowledge graph node structure and ConvE as the decoder enables the translational feature[209].”
• TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery (2022, unknown): “TransE (Bordes et al., 2013) DistMult (Yang et al., 2014) ComplEx (Trouillon et al., 2016) NeuralLP (Yang et al., 2017) SimplE (Kazemi and Poole, 2018) RotatE (Sun et al., 2019b) KBGAT (Nathani et al., 2019)”