Learning New Facts From Knowledge Bases With Neural Tensor Networks and Semantic Word Vectors

By Danqi Chen, Richard Socher, Christopher D. Manning, Andrew Y. Ng. 2013

63 citations recorded by the source corpus.

Knowledge bases provide applications with the benefit of easily accessible, systematic relational knowledge but often suffer in practice from their incompleteness and lack of knowledge of new entities and relations. Much work has focused on building or extending them by finding patterns in large unannotated text corpora. In contrast, here we mainly aim to complete a knowledge base by predicting additional true relationships between entities, based on generalizations that can be discerned in the given knowledgebase. We introduce a neural tensor network (NTN) model which predicts new relationship entries that can be added to the database. This model can be improved by initializing entity representations with word vectors learned in an unsupervised fashion from text, and when doing this, existing relations can even be queried for entities that were not present in the database. Our model generalizes and outperforms existing models for this problem, and can classify unseen relationships in WordNet with an accuracy of 75.8%.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Irreflexive and Hierarchical Relations as Translations (2013, methodology): “However, they have been outperformed by energy-based models (Bordes et al., 2011; Jenatton et al., 2012; Bordes et al., 2013; Chen et al., 2013).”
Tree-Structured Composition in Neural Networks without Tree-Structured Architectures (2015, methodology): “For the combining layer, we use a neural tensor network (NTN, Chen et al. 2013) layer, which sums the output of a plain recursive/recurrent neural network layer with a vector computed using two multiplications with a learned (full rank) third-order tensor parameter: Our model is largely identical…”
Recursive Neural Networks Can Learn Logical Semantics (2014, methodology): “The comparison layer uses the same layer function as the composition layers (either an NN layer or an NTN layer) with independently learned parameters and a separate nonlinearity function.”
NaturalLI: Natural Logic Inference for Common Sense Reasoning (2014, methodology): “Chen et al. (2013) and Socher et al. (2013) use Neural Tensor Networks to predict unseen relation triples in WordNet and Freebase, following a line of work by Bordes et al. (2011) and Jenatton et al. (2012).”
KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts (2015, methodology): “A related approach was using Neural Tensor Networks to extract new facts from an existing KB (Chen et al., 2013; Socher et al., 2013).”
Can recursive neural tensor networks learn logical reasoning? (2013, methodology): “Deep learning methods in NLP which learn vector representations for words have seen successful uses in recent years on increasingly sophisticated tasks [1, 2, 3, 4].”
Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs (2019, background): “Previous work has explored various ways of representing information in knowledge bases (Bor-des et al., 2011) and improving these representations (Chen et al., 2013).”
Analogical Inference for Multi-relational Embeddings (2017, background): “, 2011), neural tensor networks (Socher et al., 2013; Chen et al., 2013), translationbased models (Bordes et al.”
Translating Embeddings for Modeling Multi-relational Data (2013, methodology): “We detail here the links between our model and those of [3] (Structured Embeddings or SE ) and [14].”
Modeling Newswire Events using Neural Networks for Anomaly Detection (2014, background): “More recently, an alternative approach for doing the same was proposed by Chen et al. (2013).”
Combining Natural Logic and Shallow Reasoning for Question Answering (2016, background): “Knowledge base completion (e.g., Chen et al. (2013), Bordes et al. (2011), or Riedel et al. (2013)) can be thought of as entailment, predicting novel knowledge base entries from the original database.”
Modeling Relation Paths for Representation Learning of Knowledge Bases (2015, unknown): “, 2009; Zhu, 2012), energy-based models (Bordes et al., 2011; Chen et al., 2013; Socher et al., 2013; Bordes et al., 2013; Bordes et al., 2014), matrix factorization (Singh and Gordon, 2008; Nickel et al.”
Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding (2017, background): “For cross-lingual tasks, [12] extends NTNKBC [4] for cross-lingual KB completion.”