Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors

By Pentti Kanerva. 2009

1,106 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Electrical and Electronic Engineering, Hardware and Architecture, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Electrical and Electronic Engineering

In Hardware and Architecture

In Representation & Self-Supervised Learning

Citations in the corpus

Neural Turing Machines (2014, background): “In response to this criticism, neural network researchers including Hinton (Hinton, 1986), Smolensky (Smolensky, 1990), Touretzky (Touretzky, 1990), Pollack (Pollack, 1990), Plate (Plate, 2003), and Kanerva (Kanerva, 2009) investigated specific mechanisms that could support both variable-binding and”
Constrained Few-shot Class-incremental Learning (2022, background): “This critical requirement can be met by exploiting hyperdimensional computing [10, 19, 35], where classes can be represented by random high-dimensional vectors with a dimensionality in the order of thousands.”
Models of Semantic Memory (2015, methodology): “A similar approach to BEAGLE, known as random indexing, has been taken by Kanerva and colleagues (Kanerva, 2009; Kanerva, Kristoferson, & Holst, 2000).”
The Inside-Outside Recursive Neural Network model for Dependency Parsing (2014, background): “It is difﬁcult to characterise the expressivity of the resulting system in logical terms, but recent work suggests it is surprisingly powerful (e.g., Kanerva (2009)).”
Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks (2022, background): “A recent methodology [12] addresses these issues by enhancing the key memory representations with robust properties of hyperdimensional computing [13], such that the representations can be readily transformed to low-precision (i.e., bipolar or binary) vectors in the key memory while exhibiting…”
In-memory hyperdimensional computing (2019, methodology): “Equation (2) defines how a query hypervector Q is compared against each of the prototype hypervector Pi out of c classes to find the predicted class with maximum similarity.”
Associative Long Short-Term Memory (2016, background): “Since Holographic Reduced Representations may be unfamiliar to many readers, Section 2 provides a short introduction to them and to related vector-symbolic architectures (Kanerva, 2009).”
The future of electronics based on memristive systems (2018, background): “For example, inspired by the sheer sizes of neural circuits, early work in cognition explored how brains might represent concepts and their relationships as sparse vectors in a high-dimensional space 55 .”
Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures (2020, background): “…for reproducing the brain’s ability to learn from few examples, to deal with novel situations, or to change behaviors when driven by internal information processing rather than purely by ex-ternal events (Plate, 2003; Gayler, 2003; Kanerva, 2009; Lake, Ullman, Tenen-baum, & Gershman, 2017).”
Composition in Distributional Models of Semantics (2010, background): “The success of circular correlation crucially depends on the components of the n-dimensional vectors u and v being real numbers and randomly distributed with mean 0 and variance 1/n. Binary spatter codes (Kanerva, 1988, 2009) are a particularly simple form of holographic reduced representation.”
Entity Abstraction in Visual Model-Based Reinforcement Learning (2019, result): “In contrast to other methods for binding in neural networks [48–51], formulating inference as a mechanism for variable binding allows us to model uncertainty in the values of the variables.”
Linking Connectivity, Dynamics, and Computations in Low-Rank Recurrent Neural Networks. (2017, neutral): “Such mixed selectivity is a ubiquitous property of cortical neurons (Rigotti et al., 2013; Mante et al., 2013; Churchland and Shenoy, 2007), and confers additional computational properties to our networks (Kanerva, 2009).”
Computational modeling of phonetic and lexical learning in early language acquisition: Existing models and future directions (2012, neutral): “, [104]).”