A semantic matching energy function for learning with multi-relational data

By Xavier Glorot, Antoine Bordes, Jason Aaron Edward Weston, Yoshua Bengio. 2013

735 citations recorded by the source corpus.

Large-scale relational learning becomes crucial for handling the huge amounts of structured data generated daily in many application domains ranging from computational biology or information retrieval, to natural language processing. In this paper, we present a new neural network architecture designed to embed multi-relational graphs into a flexible continuous vector space in which the original data is kept and enhanced. The network is trained to encode the semantics of these graphs in order to assign high probabilities to plausible components. We empirically show that it reaches competitive performance in link prediction on standard datasets from the literature as well as on data from a real-world knowledge base (WordNet). In addition, we present how our method can be applied to perform word-sense disambiguation in a context of open-text semantic parsing, where the goal is to learn to assign a structured meaning representation to almost any sentence of free text, demonstrating that it can scale up to tens of thousands of nodes and thousands of types of relation.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Graph Learning

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, methodology): “Sampling for the negative phase is usually done by MCMC, although some unsupervised learning algorithms (Collobert & Weston, 2008; Gutmann & Hyvarinen, 2010; Bordes et al., 2013) involve “negative examples” that are sampled through simpler procedures (like perturbations of the observed input, in a s”
GSNs : Generative Stochastic Networks (2015, methodology): “sociated with x’s sampled from P(x)). Sampling for the negative phase is usually done by MCMC, although some unsupervised learning algorithms (Collobert and Weston, 2008; Gutmann and Hyvarinen, 2010; Bordes et al., 2013) involve \negative examples" that are sampled through simpler procedures (l”
Convolutional Neural Network Architectures for Matching Natural Language Sentences (2014, background): “d model on a variety of matching tasks and its superiority to competitor models. 1 Introduction Matching two potentially heterogenous language objects is central to many natural language applications [28, 2]. It generalizes the conventional notion of similarity (e.g., in paraphrase identiﬁcation [19”
Knowledge Graph Embedding by Translating on Hyperplanes (2014, methodology): “It is used as a naive baseline in (Bordes et al. 2012; 2013b).”
DKN: Deep Knowledge-Aware Network for News Recommendation (2018, methodology): “o translationbasedmethods[5,20,28,48]usedinDKN,researchershavealsoproposed many other models such as Structured Embedding [6], Latent FactorModel[19],NeuralTensorNetwork[39]andSemanticMatching Energy [4]. Recently, the knowledge graph has also been used in many applications, such as movie recommenda”
Learning Entity and Relation Embeddings for Knowledge Graph Completion (2015, methodology): “SME model (Bordes et al. 2012; 2014) aims to capture correlations between entities and relations via multiple matrix products and Hadamard product.”
Irreflexive and Hierarchical Relations as Translations (2013, methodology): “esaurus, and support automatic text analysis. Its entities (termed synsets) correspond to word senses, and relation types deﬁne lexical relations between them. We considered the data version used in (Bordes et al., 2013). Examples of triplets are ( scoreNN1, hypernym, evaluationNN1) or Irreﬂexive an”
Deep Learning of Representations: Looking Forward (2013, methodology): “strength across tasks, e.g. for multi-task and transfer learning or domain adaptation. This can be achieved by sharing embeddings or representation functions across tasks (Collobert and Weston, 2008; Bordes et al., 2013). • Manifolds: probability mass concentrates near regions that havea muchsmaller”
Embedding Entities and Relations for Learning and Inference in Knowledge Bases (2014, methodology): “…representations of entities and relations, including tensor/matrix factorization (Singh & Gordon, 2008; Nickel et al., 2011; 2012), Bayesian clustering framework (Kemp et al., 2006; Sutskever et al., 2009), and neural networks (Paccanaro & Hinton, 2001; Bordes et al., 2013a;b; Socher et al., 2013).”
Commonsense Knowledge Base Completion (2016, methodology): “…2013; Gardner et al., 2014; West et al., 2014), particularly methods based on distributed representations and neural networks (Socher et al., 2013; Bordes et al., 2013; Bordes et al., 2014a; Bordes et al., 2014b; Yang et al., 2014; Neelakantan et al., 2015; Gu et al., 2015; Toutanova et al., 2015).”
StarSpace: Embed All The Things! (2017, methodology): “TransE was shown to out-perform RESCAL (Nickel, Tresp, and Kriegel 2011), RFM (Jenatton et al. 2012), SE (Bordes et al. 2011) and SME (Bordes et al. 2014) and is considered a standard benchmark method.”
Knowledge Graph Embedding via Dynamic Mapping Matrix (2015, methodology): “Experimental results on both WN18 and FB15k are shown in Table 4.”
Deconvolutional Latent-Variable Model for Text Sequence Matching (2017, background): “ Introduction The ability to infer the degree of match between two text sequences, and determine their semantic relationship, is of central importance in natural language understanding and reasoning (Bordes et al. 2014). With recent advances in deep neural networks, considerable research has focused”
A latent factor model for highly multi-relational data (2012, methodology): “Table 1: Comparisons of the performance obtained by our approach, RESCAL [16], MRC [10] and SME [2] over three standard datasets.”
Knowledge Graph Completion via Complex Tensor Factorization (2017, background): “Motivating applications of knowledge graph completion include question answering (Bordes et al., 2014b) and more generally probabilistic querying of knowledge bases (Huang and Liu, 2009; Krompaß et al., 2014).”