Holographic reduced representations

By Tony A. Plate. 1995

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013, background): “Some related models such as holographic reduced representations (Plate, 1995), quantum logic (Widdows, 2008), discrete-continuous models (Clark and Pulman, 2007) and the recent compositional matrix space model (Rudolph and Giesbrecht, 2010) have not been experimentally validated on larger corpora.”
On the Binding Problem in Artificial Neural Networks (2020, background): “TPRs were first introduced in Smolensky (1990) and several modifications have since been proposed that consider different binding, unbinding, and conjunction operations (Plate, 1995; Kanerva, 1996; Gayler, 1998; see Kelly et al., 2013 for an overview).”
Relational inductive biases, deep learning, and graph networks (2018, background): “…were developed in domains such as analogy-making, linguistic analysis, symbol manipulation, and other forms of relational reasoning (Smolensky, 1990; Hinton, 1990; Pollack, 1990; Elman, 1991; Plate, 1995; Eliasmith, 2013), as well as more integrative theories for how the mind works (Marcus, 2001).”
Deep Learning: Methods and Applications (2014, background): “, they can potentially perform powerful reasoning over structures, as argued in various historical publications as well as recent essays [38, 156, 286, 288, 292, 336, 335].”
Constrained Few-shot Class-incremental Learning (2022, background): “This critical requirement can be met by exploiting hyperdimensional computing [10, 19, 35], where classes can be represented by random high-dimensional vectors with a dimensionality in the order of thousands.”
Superposition of many models into one (2019, background): “Plate (1995) suggest using a separate auto-associative memory for clean-up.”
Holographic Embeddings of Knowledge Graphs (2015, background): “In particular, holographic reduced representations (Plate 1995) store the association of a with b via their circular convolution”
Domain and Function: A Dual-Space Model of Semantic Relations and Compositions (2012, background): “Circular convolution is similar to the outer product, but the outer product matrix is compressed back down to a vector, c = a ⊛ b (Plate, 1995; Jones & Mewhort, 2007).”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “Some, such as vector symbolic architectures 53 , which were proposed in cognitive science (Plate, 1995; Stewart and Eliasmith, 2009; Eliasmith, 2013), are also being considered in the context of eﬃciently-trainable artiﬁcial neural networks (Danihelka et al., 2016)—in eﬀect, these systems learn how…”
Dimensions of Neural-symbolic Integration - A Structured Survey (2005, methodology): “The related approach on Holographic reduced representations (HRRs) [Plate, 1991, Plate, 1995] also uses ﬁxed-length representations of variable-sized data, but using diﬀerent methods.”
Biologically Plausible, Human-scale Knowledge Representation (2016, methodology): “Here we employ a close relative of Holographic Reduced Representations (Plate, 2003), a type of VSA in which binding is implemented via circular convolution , superposition is implemented via vector addition and vectors representing symbols are randomly chosen from the D-dimensional unit sphere.”
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations (2020, background): “If these vectors are drawn from a multivariate Gaussian distribution, the function guarantees that This operation is a circular convolution (cid:63) (as for Holographic Reduced Repre-sentations (Plate, 1995)) with a permutation matrix Φ : u ⊗ v = u ∗ Φ v .”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, methodology): “Given a fact ðh; r; tÞ, the entity representations are first composed into h ? t 2 R by using the circular correlation operation [44], namely”
Enhancing clinical concept extraction with distributional semantics (2011, methodology): “This method is an alternative implementation to the convolution operations used by BEAGLE [49] (based on Tony Plate’s Holographic Reduced Representation (HRR) [50]) to encode word-order information in word spaces.”
Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences (2019, methodology): “Considering the implicity and variety of user preferences, we design two main components in TUP: Preference Induction and Hyperplane-based Translation.”
RNNs Implicitly Implement Tensor Product Representations (2018, unknown): “Here we test two alternatives, both of which can be viewed as special cases of the tensor product or as related to it: circular convolution, which is used in holographic reduced representations (Plate, 1995), and elementwise product ( f i (cid:12) r i ).”