SORN: A Self-Organizing Recurrent Neural Network

By Andreea Lazar, Gordon Pipa, Jochen Triesch. 2009

90 citations recorded by the source corpus.

Understanding the dynamics of recurrent neural networks is crucial for explaining how the brain processes information. In the neocortex, a range of different plasticity mechanisms are shaping recurrent networks into effective information processing circuits that learn appropriate representations for time-varying sensory stimuli. However, it has been difficult to mimic these abilities in artificial neural network models. Here we introduce SORN, a self-organizing recurrent network. It combines three distinct forms of local plasticity to learn spatio-temporal patterns in its input while maintaining its dynamics in a healthy regime suitable for learning. The SORN learns to encode information in the form of trajectories through its high-dimensional state space reminiscent of recent biological findings on cortical coding. All three forms of plasticity are shown to be essential for the network's success.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

Citations in the corpus

• The temporal paradox of Hebbian learning and homeostatic plasticity (2017, neutral): “[49] Litwin-Kumar and Doiron [50] * van Rossum et al.”
• RM-SORN: a reward-modulated self-organizing recurrent neural network (2015, unknown): “Task Descriptions The model was evaluated on eight different tasks, including those in Lazar et al. (2009) and Toutounji and Pipa (2014).”
• Synaptic Plasticity in Neural Networks Needs Homeostasis with a Fast Rate Detector (2013, unknown): “This is in contrast to theoretical models that apply scaling by algorithmically enforcing weight normalization [13,18] on the timescale of one or a few simulation time-steps.”