Correlation Matrix Memories

By Teuvo Kohonen. 1972

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Locating and Editing Factual Associations in GPT (2022, methodology): “We view W (l) proj as a linear associative memory (Kohonen, 1972; Anderson, 1972).”
Mass-Editing Memory in a Transformer (2022, background): “Previously, Dai et al. (2022) and Yao et al. (2022) have proposed editing methods that alter sparse sets of neurons, but we adopt the classical view of a linear layer as an associative memory (Anderson, 1972; Kohonen, 1972).”
Deep learning in neural networks: An overview (2014, background): “…brought simple NNs trained by SL (e.g., Narendra & Thathatchar, 1974; Rosenblatt, 1958, 1962; Widrow & Hoff, 1962) and UL (e.g., Grossberg, 1969; Kohonen, 1972; von der Malsburg, 1973; Willshaw & von der Malsburg, 1976), as well as closely related associative memories (e.g., Hopfield, 1982;…”
Using Fast Weights to Attend to the Recent Past (2016, background): “One of the main preoccupations of neural network research in the 1970s and early 1980s [Willshaw et al., 1969, Kohonen, 1972, Anderson and Hinton, 1981, Hopﬁeld, 1982] was the idea that memories were not stored by somehow keeping copies of patterns of neural activity.”
Automatically Composing Representation Transformations as a Means for Generalization (2018, background): “The fast-weights (FW) literature [7, 71] is also related to the idea of selecting different weights at different steps of computation, but with a different motivation for learning context-dependent associative memory [3, 39, 44, 48, 86].”
Synaptic Plasticity Forms and Functions. (2020, background): “Here, inputs driving a given output will have their connection strengths enhanced,forming a simple association that allows even fragments of the associated input pattern to evoke the correct output activity (Andersen 1972, Kohonen 1972, Hopfield 1982).”
Synchronization and pattern separation in spiking associative memories and visual cortical areas (2004, methodology): “, [103, 104]) where per definition the result is the linear superposition of single input components.”
Moving the Eiffel Tower to ROME: Tracing and Editing Facts in GPT (2021, background): “The linear associative memory view of a single 326 linear layer is a classical model (Kohonen, 1972; 327 Anderson, 1972) that notes that any linear operation”
Cocaine and Chest Pain (1992, unknown): “Notable exceptions from this period are the works of Amari [24], Anderson [25], Fukushima [26], Grossberg [27] and Kohonen [28].”
Linear Transformers Are Secretly Fast Weight Programmers (2021, background): “…memory can be found in numerous works since Hebb’s informal rule (Hebb, 1949) and its more concrete formal variants (Steinbuch, 1961; Steinbuch & Piske, 1963; Kohonen, 1972; Palm, 1980) including Hopﬁeld networks (Hopﬁeld, 1982; Little, 1974) and bi-directional associative nets (Kosko, 1988).”
Locating and Editing Factual Knowledge in GPT (2022, methodology): “To reason about these structures, we view W ( l ) proj as a linear associative memory (Kohonen, 1972; Anderson, 1972).”
Optimal Plasticity from Matrix Memories: What Goes Up Must Come Down (1990, methodology): “In principle, the association between the synchronous activities in two neurons could be registered by a mechanism that increases the efficacy of the synapses between them, in the manner first proposed by Hebb (1949); the generalization of this idea to the storage of the associations between activit”
A History of Spike-Timing-Dependent Plasticity (2011, background): “The work of Steinbuch inspired Teuvo Kohonen, who cites Steinbuch in his article on correlation matrix memories (Kohonen, 1972).”
Memory for events and their spatial context: models and experiments. (2001, background): “In this process, region CA3 acts anàuto-associative memory' in which stored patterns of activity are attractor states such that the system will return to the most similar stored pattern from any initial state (Willshaw et al. 1969; Kohonen 1972; Gardner-Medwin 1976; Hop¢eld 1982).”