Learning Distributed Representations of Concepts Using Linear Relational Embedding

By Alberto Paccanaro, Geoffrey E. Hinton. 2001

142 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Neural Execution Engines: Learning to Execute Subroutines (2020, background): “For example, [17] directly embeds integers in the range [ − 10 , 10] as vectors and trains these, along with matrices representing relationships between objects.”
How to Represent Part-Whole Hierarchies in a Neural Network (2021, background): “To prevent collapse, contrastive learning [Becker and Hinton, 1992, Paccanaro and Hinton, 2001, van den Oord et al., 2018] uses negative examples and tries to make representations that should agree be close while maintaining separation between representations which should not agree 21 .”
A semantic matching energy function for learning with multi-relational data (2013, methodology): “More precisely, it is connected to models aiming at composing such vector embeddings for obtaining phrase or context representations (Mitchell and Lapata 2008), via tensor products (Smolensky 1990) or matrix operations (Paccanaro and Hinton 2001; Socher et al. 2012).”
Learning Structured Embeddings of Knowledge Bases (2011, methodology): “Linear Relational Embeddings (Paccanaro and Hinton 2001) are also inspiring for this work but with a different loss function and architecture and have onl y been applied to small arithmetic and “family relation” task s.”
Embedding Entities and Relations for Learning and Inference in Knowledge Bases (2014, background): “Such large-scale multi-relational data provide an excellent potential for improving a wide range of tasks, from information retrieval, question answering to biological data mining.”
Barnes-Hut-SNE (2013, methodology): “In addition, we plan to explore to what extent adapted versions of our algorithm (that use metric trees instead of quadtrees) can be used to speed up techniques for relational embedding ( e.g. , [4, 18]).”
Unsupervised Grounding of Plannable First-Order Logic Representation from Images (2019, background): “e demonstrate the ability of FOSAE on a photo-realistic Blocksworld domain. 2 Related Work One of the early work on learning the relations between object symbols includes Linear Relational Embedding (Paccanaro and Hinton 2001), which is learned from the tuples of relations and objects in a supervise”
Heterogeneous Network Embedding via Deep Architectures (2015, methodology): “Unlike to traditional linear embedding models [29, 44, 50], the proposed method decomposes the feature learning process into multiple nonlinear layers of a deep architecture.”
Structured Embedding via Pairwise Relations and Long-Range Interactions in Knowledge Base (2015, methodology): “A similar algorithm is utilized to represent concepts as vectors and relations as matrices in Linear Relation Embedding (LRE) (Paccanaro 2000; Paccanaro and Hinton 2001) which maps entities into a lowdimensional space by imposing the constraint that relations in this low-dimensional space are…”
Learning to Represent Knowledge Graphs with Gaussian Embedding (2015, background): “LRE (2001) [20] exp(−||Wrh− t||), Wr ∈ Rke×ke kene + krnr SE (2011) [5] ||Wrhh−Wrtt||`1/2 , Wrh,Wrt ∈ R kr×ke kene + 2krnr”
A latent factor model for highly multi-relational data (2012, background): “Another natural extension to learning several relations simultaneously can be to share the common embedding or the entities across relations via collective matrix factorization as proposed in RESCAL [15] and other related work [18, 23].”
Poincaré Embeddings for Learning Hierarchical Representations (2017, methodology): “For instance, in addition to the methods discussed in Section 1, Paccanaro and Hinton [26] proposed one of the first embedding methods to learn from relational data.”
Using matrices to model symbolic relationship (2008, methodology): “Paccanaro and Hinton [10] introduced a method called “Linear Relational Embedding” (LRE) that uses multiplication of vectors by matrices in the target domain to model pairwise relations between objects in the data domain.”
Knowledge Graph Completion: A Review (2020, background): “For multi-relational data, linear relation embedding [73], [74] imposes constraints, that is, through linear operations to model the relationship in this feature space, thereby learning the mapping from the entity to the feature space.”
Joint Learning of Words and Meaning Representations for Open-Text Semantic Parsing (2012, background): “Paccanaro and Hinton (2001) developed entity embedding for learning logical relations (but do not model words).”