Traversing Knowledge Graphs in Vector Space
By Kelvin Guu, John Miller, Percy Liang. 2015
367 citations recorded by the source corpus.
Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts path queries. Recent models for knowledge base completion impute missing facts by embedding knowledge graphs in vector spaces. We show that these models can be recursively applied to answer path queries, but that they suffer from cascading errors. This motivates a new "compositional" training objective, which dramatically improves all models' ability to answer path queries, in some cases more than doubling accuracy. On a standard knowledge base completion task, we also demonstrate that compositional training acts as a novel form of structural regularization, reliably improving performance across all base models (reducing errors by up to 43%) and achieving new state-of-the-art results.
Subfields: Artificial Intelligence, Knowledge Graphs & Representation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Percy Liang
- Yoshua Bengio
- Jianfeng Gao
- Luke S. Zettlemoyer
- Quoc V. Le
- Yejin Choi
- Kyunghyun Cho
In Knowledge Graphs & Representation
- Sebastian Riedel
- Timothée Lacroix
- Zhiyuan Liu
- Maosong Sun
- Tim Rocktaschel
- Nicolas Usunier
- Guillaume Obozinski
- William Yang Wang
Citations in the corpus
- FiLM: Visual Reasoning with a General Conditioning Layer (2017, methodology): “However, approaches from word embeddings, representation learning, and zero-shot learning can be applied to directly optimize ( γ , β ) for analogy-making (Bordes et al. 2013; Guu, Miller, and Liang 2015; Oh et al. 2017).”
- Neural Approaches to Conversational AI (2018, neutral): “, “Where did Tad Lincoln’s parents live?” (Toutanova et al., 2016; Guu et al., 2015; Neelakantan et al., 2015).”
- Data Recombination for Neural Semantic Parsing (2016, background): “Guu et al. (2015) showed that encouraging a knowledge base completion model to handle longer path queries acts as a form of structural regularization.”
- The More You Know: Using Knowledge Graphs for Image Classification (2016, methodology): “Using the subset of WordNet from [10], we first collect new nodes in WordNet not in our output label by including those which directly connect to our output labels and thus likely to be relevant and add them to a combined graph.”
- Modeling Relational Data with Graph Convolutional Networks (2017, methodology): “We can roughly classify previous work into (1) methods creating auxiliary triples, which are then added to the learning objective of a factorization model [14, 12]; (2) approaches using paths (or walks) as features when predicting edges [23]; or (3) doing both at the same time [25, 36].”
- Commonsense Knowledge Base Completion (2016, methodology): “…2013; Gardner et al., 2014; West et al., 2014), particularly methods based on distributed representations and neural networks (Socher et al., 2013; Bordes et al., 2013; Bordes et al., 2014a; Bordes et al., 2014b; Yang et al., 2014; Neelakantan et al., 2015; Gu et al., 2015; Toutanova et al., 2015).”
- Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning (2016, methodology): “Recent work has explored several techniques to perform Knowledge Base Completion (KBC) such as vector space models and graph traversal (Socher et al., 2013; Yang et al., 2014; Gardner et al., 2014; Neelakantan et al., 2015; Guu et al., 2015).”
- A Neural Knowledge Language Model (2016, background): “ich is an extension of word embedding techniques [4] from neural language models, provide distributed representations for the entities in the KG. In addition, the graph can be traversed for reasoning [15]. Finally, facts come along with a textual representations which we call the fact description an”
- Holographic Embeddings of Knowledge Graphs (2015, background): “Furthermore, Guu, Miller, and Liang (2015) proposed a R ESCAL -based model to learn from paths in KGs. Smolensky (1990) introduced the tensor product as a way to create compositional vector representations.”
- Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks (2016, methodology): “In comparison with previous state-of-the-art (Guu et al., 2015) our model achieves a 84% reduction in error in mean quantile.”
- Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, background): “, entity types [25], [26], relation paths [27], [28], [29], textual descriptions [30], [31], [32], [33], and even logical rules [23], [34], [35], to learn more pre-”
- Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text (2016, methodology): “11 Model MAP HITS@10 B ILINEAR - DIAG (Guu et al., 2015) N/A in (Guu et al., 2015) and our implementation.”
- Learning Knowledge Base Inference with Neural Theorem Provers (2016, background): “Gu et al. (2015) traverse KBs in vector space to answer queries.”
- Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings (2020, methodology): “The entire procedure is analogous to traversing KGs following the computation graph (Guu et al., 2015).”
- Representation Learning of Knowledge Graphs with Hierarchical Types (2016, methodology): “Inspired by [Hu et al., 2015], each sub-type (i.e. layer) in hierarchy is represented as a projection matrix with different granularities.”
- A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network (2017, background): “tities in the KBs could help to get contextual information for improving KB completion performance (Lin et al., 2015a; Luo et al., 2015; Guu et al., 2015; Toutanova et al., 2016; Nguyen et al., 2016a).”
- Learning Sequence Encoders for Temporal Knowledge Graph Completion (2018, background): “Most approaches to link prediction are characterized by a scoring function that operates on the entity and relation type embeddings of a triple (Bordes et al., 2013; Yang et al., 2014; Guu et al., 2015).”
- Representation Learning for Visual-Relational Knowledge Graphs (2017, background): “Examples are knowledge base factorization and embedding approaches [10,11,12,13] and random-walk based ML models [14,15].”
- Leveraging Knowledge Bases in LSTMs for Improving Machine Reading (2017, methodology): “In parallel to the development for text processing, neural networks have been successfully used to learn distributed representations of structured knowledge from large KBs (Bordes et al., 2011, 2013; Socher et al., 2013; Yang et al., 2015; Guu et al., 2015).”
- SSE: Semantically Smooth Embedding for Knowledge Graphs (2017, background): “This section briefly reviews two lines of related work: KG embedding and manifold learning.”