Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements

By Shun-ichi Amari. 1972

Various information-processing capabilities of self-organizing nets of threshold elements are studied. A self-organizing net, learning from patterns or pattern sequences given from outside as stimuli, "remembers" some of them as stable equilibrium states or state-transition sequences of the net. A condition where many patterns and pattern sequences are remembered in a net at the same time is shown. The stability degree of their remembrance and recalling under noise disturbances is investigated theoretically. For this purpose, the stability of state transition in an autonomous logical net of threshold elements is studied by the use of characteristics of threshold elements.

Subfields: Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, background): “Hopﬁeld networks are energy-based, binary associative memories, which popularized artiﬁcial neural networks in the 1980s (Amari, 1972; Hopﬁeld, 1982, 1984).”
Colloquium: Criticality and dynamical scaling in living systems (2017, background): “It was long-ago conjectured that the extraordinary “computational power” of living systems could be the result of collective behavior, emerging out of a large number of simple components (Amari, 1972; Carpenter and Grossberg, 2016; Grossberg, 1982; Hopfield, 1982).”
Matching Recall and Storage in Sequence Learning with Spiking Neural Networks (2013, background): “Learning to autonomously reproduce a given spatio-temporal activity pattern is a fundamental challenge approached by the earliest models of recurrent neural networks (Amari, 1972; Hopﬁeld, 1982; Herz et al., 1988; Gerstner et al., 1993; Horn et al., 2000).”
Using Conceptors to Manage Neural Long-Term Memories for Temporal Patterns (2017, background): “This leads to recall episodes which step through a discrete sequence of patterns (Amari, 1972). The basic discrete hetero-associative memory mechanism has been elaborated in various ways to accomodate analog values, smoother temporal development, or identical patterns at different time slices (examp”
An ultra-sparse code underliesthe generation of neural sequences in a songbird (2002, background): “In most neural models of sequence generation, a sequence of activity is stored by specific excitatory and inhibitory connections between neurons within the network, such that the network advances autonomously from one state to the nex”