Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction

By Jason Aaron Edward Weston, Antoine Bordes, Oksana Yakhnenko, Nicolas Usunier. 2013

250 citations recorded by the source corpus.

This paper proposes a novel approach for relation extraction from free text which is trained to jointly use information from the text and from existing knowledge. Our model is based on scoring functions that operate by learning low-dimensional embeddings of words, entities and relationships from a knowledge base. We empirically show on New York Times articles aligned with Freebase relations that our approach is able to efficiently use the extra information provided by a large subset of Freebase data (4M entities, 23k relationships) to improve over methods that rely on text features alone.

Subfields: Artificial Intelligence, Molecular Biology, Information Extraction, Knowledge Graphs & Representation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Information Extraction

In Knowledge Graphs & Representation

Citations in the corpus

Knowledge Graph Embedding by Translating on Hyperplanes (2014, methodology): “For a clear and fair comparison with TransE reported in (Weston et al. 2013), we implement the same text side method Wsabie M2R in (Weston et al.”
Learning Entity and Relation Embeddings for Knowledge Graph Completion (2015, methodology): “In (Weston et al. 2013), TransE and text-based extraction model were combined to rank candidate facts and achieved promising improvement.”
Open Question Answering with Weakly Supervised Embedding Models (2014, background): “Contrary to previous work modeling KBs with embeddings (e.g. [24]), in our model, an entity does not have the same embedding when appearing in the left-hand or in the right-hand side of a triple.”
Representing Text for Joint Embedding of Text and Knowledge Bases (2015, background): “Weston et al. (2013) combined continuous representations from a knowledge base and textual mentions for prediction of new relations.”
StarSpace: Embed All The Things! (2017, methodology): “• Link Prediction in Knowledge Bases: we show that our method outperforms several methods, including transE (Bordes et al. 2013) on Freebase 15K. • Wikipedia Search and Sentence Matching tasks: it out-performs off-the-shelf embedding models due to directly training sentence and document-level…”
Knowledge Graph Embedding via Dynamic Mapping Matrix (2015, methodology): “Among these methods, TransE (Bordes et al. 2013) is simple and effective, and also achieves state-of-the-art prediction performance.”
Distributional vectors encode referential attributes (2015, methodology): “Our proposal is only distantly related to methods to embed words tokens and KB entities and relationships in a vector space, e.g., for better relation extraction (see Weston et al. (2013) and references therein).”
Translating Embeddings for Modeling Multi-relational Data (2013, methodology): “Hence, we recently fruitfully inserted TransE into a framework for relation extraction from text [16].”
Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions (2017, background): “Another line is to introduce external semantic repositories such as knowledge graphs (Weston et al. 2013; Wang et al. 2014).”
Discrete-State Variational Autoencoders for Joint Discovery and Factorization of Relations (2016, background): “…completion via matrix or tensor factorization has received a lot of attention in the past few years (Bordes et al., 2011; Jenatton et al., 2012; Weston et al., 2013; Bordes et al., 2013; Socher et al., 2013; Garcı́a-Durán et al., 2014; Bordes et al., 2014; Lin et al., 2015b; Chang et al.,…”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, background): “Those entity and relation embeddings can further be used to benefit all kinds of tasks, such as KG completion [14], [15], relation extraction [20], [21], entity classification [13], [22], and entity resolution [13], [18].”
Semantically Smooth Knowledge Graph Embedding (2015, background): “The learned embeddings can further be used to beneﬁt all kinds of tasks, such as KG completion (Socher et al., 2013; Bordes et al., 2013), relation extraction (Riedel et al., 2013; Weston et al., 2013), and entity resolution (Bordes et al., 2014).”
A benchmarking study of embedding-based entity alignment for knowledge graphs (2020, unknown): “KG embeddings have been successfully applied to many tasks such as link prediction [5] and relation extraction [91].”
Modeling Relation Paths for Representation Learning of Knowledge Bases (2015, methodology): “TransE was used to enrich a text-based model and achieved a signiﬁcant improvement (Weston et al., 2013), and so do TransH (Wang et al., 2014) and TransR (Lin et al., 2015).”
Aligning Knowledge and Text Embeddings by Entity Descriptions (2015, methodology): “We use the same public dataset NYT+FB, released by Riedel et al. (2010) and used in (Weston et al., 2013) and (Wang et al., 2014a).”
SSE: Semantically Smooth Embedding for Knowledge Graphs (2017, background): “Index Terms—Knowledge graph embedding, semantic smoothness, Laplacian eigenmaps, locally linear embedding Ç”
Relation Extraction Using Distant Supervision (2018, background): “The authors of [66] propose an embedding model that is a composition of two submodels, one of which learns embeddings from text while the other one learns from the knowledge base.”