Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema

By Andrew McCallum, Arvind Neelakantan, Pat Verga. 2016

48 citations recorded by the source corpus.

Universal schema predicts the types of entities and relations in a knowledge base (KB) by jointly embedding the union of all available schema types—not only types from multiple structured databases (such as Freebase or Wikipedia infoboxes), but also types expressed as textual patterns from raw text. This prediction is typically modeled as a matrix completion problem, with one type per column, and either one or two entities per row (in the case of entity types or binary relation types, respectively). Factorizing this sparsely observed matrix yields a learned vector embedding for each row and each column. In this paper we explore the problem of making predictions for entities or entity-pairs unseen at training time (and hence without a pre-learned row embedding). We propose an approach having no per-row parameters at all; rather we produce a row vector on the fly using a learned aggregation function of the vectors of the observed columns for that row. We experiment with various aggregation functions, including neural network attention models. Our approach can be understood as a natural language database, in that questions about KB entities are answered by attending to textual or database evidence. In experiments predicting both relations and entity types, we demonstrate that despite having an order of magnitude fewer parameters than traditional universal schema, we can match the accuracy of the traditional model, and more importantly, we can now make predictions about unseen rows with nearly the same accuracy as rows available at training time.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Zero-Shot Relation Extraction via Reading Comprehension (2017, background): “Verga et al. (2017) predict facts for entity pairs that were not observed in the original matrix; this is equivalent to extracting seen relation types with unseen entities (see Section 6.1).”
Neural Compositional Denotational Semantics for Question Answering (2018, methodology): “To model the large number of entities in KGs such as Freebase, techniques proposed by recent work (Verga et al., 2017; Gupta et al., 2017) that explore representing entities as composition of its properties, such as, types, description etc.”
Continuous and Interactive Factual Knowledge Learning in Veriﬁcation Dialogues (2020, background): “There are also works on KB question-answering (KBQA) [22, 7, 4, 42, 41], open-vocabulary semantic parsing [15, 47], information extraction [43, 2], and KB updating using facts extracted from the Web [35].”
Entity Linking via Joint Encoding of Types, Descriptions, and Context (2017, background): “…learn entity embeddings for other applications have also been proposed, such as, from the structured KB for KB completion (Bordes et al., 2011, 2013; Yang et al., 2014; Lin et al., 2015), or from both structured KBs, and text for relation extraction (Toutanova et al., 2016; Verga et al., 2016a).”
A Primal Dual Formulation For Deep Learning With Constraints (2019, result): “This is also observed in earlier work (Xu et al. [2018]; Mehta et al.”