Sparse Distributed Memory

By Pentti Kanerva. 1988

Subfields: Knowledge Graphs & Representation, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Knowledge Graphs & Representation

In Neuroscience

Citations in the corpus

The Arcade Learning Environment: An Evaluation Platform for General Agents (2012, methodology): “Such methods include polynomial bases (Schweitzer & Seidmann, 1985), sparse distributed memories (Kanerva, 1988) and locality sensitive hashing (LSH) (Gionis et al.”
Measuring Catastrophic Forgetting in Neural Networks (2017, methodology): “The Sparse Distributed Memory (SDM) is a convolution-correlation model that uses sparsity to reduce the overlap between internal representations (Kanerva 1988).”
Re-evaluating circuit mechanisms underlying pattern separation (2019, neutral): “…the computations required for these functions are achieved (Figure 2A; Braitenberg, 1961; Eccles et al., 1967; Marr, 1969; Albus, 1971; Fujita, 1982; Pellionisz and Llinas, 1982; Kanerva, 1988; Tyrrell and Willshaw, 1992; Miall et al., 1993; Ito, 2006; Yamazaki and Tanaka, 2007; Dean et al., 2010).”
A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information (1993, background): “neural networks (e.g., Hopfield, 1982; Kanerva, 1988).”
The Utility of Sparse Representations for Control in Reinforcement Learning (2018, background): “In fact, many hand-crafted features are sparse representations, including tile coding (Sutton 1996; Sutton and Barto 1998), radial basis functions and sparse distributed memory (Kanerva 1988; Ratitch and Precup 2004).”
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting (2004, background): “…use non-overlapping, or sparsely distributed, representations face this dilemma to a considerably lesser extent (e.g. Hintzman 1986, Grossberg 1987, Kanerva 1988, Kruschke 1992, 1993, Ans et al. 1998), good generalization often requires a highly distributed system; and along with this high degree…”
Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma (1997, methodology): “, CHARM (Metcalfe, 1982) and TODAM (Murdock, 1983)), and Sparse Distributed Memory (SDM) (Kanerva, 1989) can learn new information in a sequential manner and can, in addition, generalize on new input.”
A Foundational Architecture for Artificial General Intelligence (2007, methodology): “Both episodic memories are computationally implemented using a modified sparse distributed memory ([43], [44], [45]).”
Recurrent Memory Array Structures (2016, methodology): “In the first implmentation, the algorithm randomly chooses index of the memory cell to be used.”
Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down (1990, background): “In rule R3, which Stanton and Sejnowski (1989) proposed for the mossy fibers in the hippocampal CA3 region, and which is also used in theoretical schemes (Kanerva 1988), depression is purely”
Using Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks (1991, background): “Sparse Distributed Memory (hereafter, SDM [Kanerva 1988]) is auto-associative, content-addressable memory typically consisting of one million 1000-bit "memory locations".”
Imaging a Population Code for Odor Identity in the Drosophila Mushroom Body (2013, background): “Theoretical studies suggest that sparseness is useful for accurate learning because it minimizes the overlap between response patterns evoked by different stimuli (Kanerva, 1988; Fiete et al., 2004).”
A Simple Connectivity Scheme for Sparse Coding in an Olfactory System (2007, background): “” Early work by Attneave (1954), Barlow (1969), and Marr (1969), as well as more recent theoretical work (Willshaw and Longuet-Higgins, 1970; Kanerva, 1988; Tsodyks and Feigel’man, 1988; Perez-Vicente and Amit, 1989; Garcia-Sanchez and Huerta, 2003; Huerta et al., 2004; Olshausen and Field, 2004) no”
Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations (1991, background): “Kanerva networks Kanerva, 1988] have suucient capacity, b u t can only store binary vectors.”