Compositional Vector Space Models for Knowledge Base Completion

By Arvind Neelakantan, Benjamin Roth, Andrew McCallum. 2015

299 citations recorded by the source corpus.

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic feature, like bornIn(X,Z)→ containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recurrent neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assemble a new dataset of over 52M relational triples, and show that our method improves over a traditional classifier by 11%, and a method leveraging pre-trained embeddings by 7%.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Modeling Relational Data with Graph Convolutional Networks (2017, methodology): “We can roughly classify previous work into (1) methods creating auxiliary triples, which are then added to the learning objective of a factorization model [14, 12]; (2) approaches using paths (or walks) as features when predicting edges [23]; or (3) doing both at the same time [25, 36].”
End-to-end Differentiable Proving (2017, background): “In addition, NTPs are related to random walk [48, 49, 11, 12] and path encoding models [14, 16].”
Commonsense Knowledge Base Completion (2016, methodology): “…2013; Gardner et al., 2014; West et al., 2014), particularly methods based on distributed representations and neural networks (Socher et al., 2013; Bordes et al., 2013; Bordes et al., 2014a; Bordes et al., 2014b; Yang et al., 2014; Neelakantan et al., 2015; Gu et al., 2015; Toutanova et al., 2015).”
Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning (2016, methodology): “Recent work has explored several techniques to perform Knowledge Base Completion (KBC) such as vector space models and graph traversal (Socher et al., 2013; Yang et al., 2014; Gardner et al., 2014; Neelakantan et al., 2015; Guu et al., 2015).”
Traversing Knowledge Graphs in Vector Space (2015, background): “An elegant solution to incompleteness is using vector space representations: Controlling the dimensionality of the vector space forces generalization to new facts (Nickel et al., 2011; Nickel et al., 2012; Socher et al., 2013; Riedel et al., 2013; Neelakantan et al., 2015).”
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language (2019, background): “An area in which neural multi-hop reasoning models have been investigated is Knowledge Base Completion (KBC) (Das et al., 2016; Cohen, 2016; Neelakantan et al., 2015; Rocktäschel and Riedel, 2017; Das et al., 2017; Evans and Grefenstette, 2018).”
Neural Programmer: Inducing Latent Programs with Gradient Descent (2015, methodology): “Neelakantan et al. (2015); Lin et al. (2015) learn simple Horn clauses in a large knowledge base using RNNs. Neural networks have also been used for Q&A on datasets that do not require complicated arithmetic and logic reasoning (Bordes et al., 2014; Iyyer et al., 2014; Sukhbaatar et al., 2015; Peng…”
RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space (2018, background): “There are also a large body of relational approaches for modeling the relational patterns on knowledge graphs (Lao et al., 2011; Neelakantan et al., 2015; Das et al., 2016; Rockt ¨ aschel & Riedel, 2017; Yang et al., 2017).”
Representation Learning of Knowledge Graphs with Hierarchical Types (2016, background): “It significantly improves the ability of KGs in cognition and reasoning [Bordes et al., 2013; Yang et al., 2014; Dong et al., 2014; Neelakantan et al., 2015].”
Composing Relationships with Translations (2015, background): “The idea of compositionality to model multi-relational data was previously introduced in (Neelakantan et al., 2015).”
Adversarial Sets for Regularising Neural Link Predictors (2017, background): “…tasks such as link prediction or knowledge base population [Nickel et al., 2012, Riedel et al., 2013b, Socher et al., 2013, Chang et al., 2014, Toutanova et al., 2015, Neelakantan et al., 2015], where objects can interact with each other, such goals can also be deﬁned in terms of several instances.”
Quaternion Knowledge Graph Embeddings (2019, background): “Composition patterns are commonplace in knowledge graphs [Lao et al., 2011, Neelakantan et al., 2015].”
Representing Text for Joint Embedding of Text and Knowledge Bases (2015, unknown): “Co-occurrence based textual relation representations were also learned in (Neelakantan et al., 2015).”
Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks (2017, methodology): “ence on the KB using text based features (Krishnamurthy and Mitchell, 2012; Reddy et al., 2014; Joshi et al., 2014; Yao and Van Durme, 2014; Yih et al., 2015; Neelakantan et al., 2015b; Guu et al., 2015; Xu et al., 2016b; Choi et al., 2015; Savenkov and Agichtein, 2016) or improves the inference on ”
Learning from positive and unlabeled data: a survey (2018, methodology): “The reason why this assumption is popular is because it enables the use of standard machine learning methods for supervised binary classification (Neelakantan et al. 2015).”
Lifted Rule Injection for Relation Embeddings (2016, methodology): “Current successful methods for automated knowledge base construction tasks heavily rely on learned distributed vector representations (Nickel et al., 2012; Riedel et al., 2013; Socher et al., 2013; Chang et al., 2014; Neelakantan et al., 2015; Toutanova et al., 2015; Nickel et al., 2015; Verga et al”
Constructing Datasets for Multi-hop Reading Comprehension Across Documents (2017, methodology): “parsity problems by introducing synthetic links via dense latent embeddings. Several other methods have been proposed, using composition functions such as vector addition (Bordes et al., 2014), RNNs (Neelakantan et al., 2015; Das et al., 2017), and memory networks (Jain, 2016). Another approach is t”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “Path-RNN [164] recursively aggregates relation path features for multi-hop reasoning.”
Variational Reasoning for Question Answering with Knowledge Graph (2017, background): “Multi-hop reasoning: There are some other works on knowledge graph completion with traversal, which requires path sampling [12, 31] or dynamic programming [32].”