Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text

By Kristina Toutanova, Xi Victoria Lin, Wen-tau Yih, Hoifung Poon, Chris Quirk. 2016

155 citations recorded by the source corpus.

Modeling relation paths has offered significant gains in embedding models for knowledge base (KB) completion. However, enumerating paths between two entities is very expensive, and existing approaches typically resort to approximation with a sampled subset. This problem is particularly acute when text is jointly modeled with KB relations and used to provide direct evidence for facts mentioned in it. In this paper, we propose the first exact dynamic programming algorithm which enables efficient incorporation of all relation paths of bounded length, while modeling both relation types and intermediate nodes in the compositional path representations. We conduct a theoretical analysis of the efficiency gain from the approach. Experiments on two datasets show that it addresses representational limitations in prior approaches and improves accuracy in KB completion.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Modeling Relational Data with Graph Convolutional Networks (2017, methodology): “…objec-tive of a factorization model (Guu, Miller, and Liang 2015; Garcia-Duran, Bordes, and Usunier 2015); (2) approaches using paths (or walks) as features when predicting edges (Lin et al. 2015); or (3) doing both at the same time (Nee-lakantan, Roth, and McCallum 2015; Toutanova et al. 2016).”
Lightly-supervised Representation Learning with Global Interpretability (2018, background): “Distributed representations of words (Mikolov et al., 2013; Levy and Goldberg, 2014a) serve as underlying representation for many NLP tasks such as information extraction and question answering (Riedel et al., 2013; Toutanova et al., 2015, 2016; Sharp et al., 2016).”
Variational Reasoning for Question Answering with Knowledge Graph (2017, background): “Multi-hop reasoning: There are some other works on knowledge graph completion with traversal, which requires path sampling (Guu, Miller, and Liang 2015; Neelakantan, Roth, and McCallum 2015) or dynamic programming (Toutanova et al. 2016).”
Entity Linking via Joint Encoding of Types, Descriptions, and Context (2017, background): “, 2015), or from both structured KBs, and text for relation extraction (Toutanova et al., 2016; Verga et al., 2016a).”
Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction (2021, methodology): “However, All-Paths is restricted to bilinear models and has limited model capacity.”
Neural Approaches to Conversational AI (2018, unknown): “rparameters, and Dthe training set of triples. These basic KB models have been extended to answer multi-step relation queries, as known as path queries, e.g., “Where did Tad Lincoln’s parents live?” (Toutanova et al., 2016; Guu et al., 2015; Neelakantan et al., 2015). A path query consists of an ini”
Multi-Hop Knowledge Graph Reasoning with Reward Shaping (2018, unknown): “Early work treats it as a link prediction problem and perform maximum-likelihood classiﬁcation over either discrete path features (Lao et al., 2011, 2012; Gardner et al., 2013) or their hidden representations in a vector space (Guu et al., 2015; Toutanova et al., 2016; Das et al., 2017).”
Learning Cross-Context Entity Representations from Text (2020, background): “…of work in learning representations of entities by building knowledge base embeddings (Bordes et al., 2011; Socher et al., 2013; Yang et al., 2014; Toutanova et al., 2016; Vilnis et al., 2018), and by jointly embedding knowledge bases and information from textual mentions (Riedel et al., 2013;…”
Deep Biaffine Attention for Neural Dependency Parsing (2016, methodology): “We use a larger but more thoroughly regularized parser than other recent BiLSTM-based approaches, with biaffine classifiers to predict arcs and labels.”
Deep Contextualized Self-training for Low Resource Dependency Parsing (2019, background): “Our focus task is dependency parsing, which is essential for many NLP tasks (Levy and Goldberg, 2014; Angeli et al., 2015; Toutanova et al., 2016; Hadiwinoto and Ng, 2017; Marcheggiani et al., 2017), but where self-training has typically failed (§ 2).”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “, entity types [25], [26], relation paths [27], [28], [29], textual descriptions [30], [31], [32], [33], and even logical rules [23], [34], [35], to learn more pre-”
Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning (2017, background): “…those found by following inference paths in KBs. Multi-hop link prediction approaches (Lao et al., 2011; Neelakantan et al., 2015; Guu et al., 2015; Toutanova et al., 2016; Das et al., 2017) address the problems above, but the reasoning paths that they operate on are gathered by (i) Can learn…”
A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network (2017, background): “tities in the KBs could help to get contextual information for improving KB completion performance (Lin et al., 2015a; Luo et al., 2015; Guu et al., 2015; Toutanova et al., 2016; Nguyen et al., 2016a).”