Long short-term memory and Learning-to-learn in networks of spiking neurons

By Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert A. Legenstein, Wolfgang Maass. Neural Information Processing Systems, 2018

601 citations recorded by the source corpus.

Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain. But computing and learning capabilities of RSNN models have remained poor, at least in comparison with artificial neural networks (ANNs). We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT). A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture). In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (L2L) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples. We demonstrate this for supervised learning and reinforcement learning.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/80eec7deeffd76a42db351409d85e4a9165542b0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (2024, methodology): “A.2 BPTT WITH SG Backpropagation through time (BPTT) with surrogate gradients (SG) is a popular way to train SNNs without specific coding schemes (Shrestha & Orchard, 2018; Bellec et al., 2018; Wu et al., 2018; Zheng et al., 2021; Li et al., 2021; Fang et al., 2021; Yin et al., 2021).”
The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks (2020, methodology): “…(Bohte, 2011), train convolutional SNNs on challenging neuromorphic (Amir et al., 2017; Orchard, Jayawant, Cohen, & Thakor, 2015) and vision benchmarks (Esser et al., 2016), or train recurrent SNNs on temporal problems requiring working memory (Bellec et al., 2018; Shrestha & Orchard, 2018).”
A review on the long short-term memory model (2020, background): “The authors in Bellec et al. (2018) built upon recurrent networks of spiking neurons, developing Long short-term memory Spiking Neural Networks (LSNN) including adapting neurons.”
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE) (2018, neutral): “BPTT for SNNs was investigated in [4, 12, 21, 3, 10].”
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network (2020, methodology): “In order to scale the networks much deeper, spike-based error backpropagation algorithms have been proposed for the supervised training of SNNs [1, 15, 19, 21, 27, 29, 35, 41].”
Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks (2019, methodology): “rogate derivatives have also been used to train spiking RCNNs where dynamical recurrence arises due to the use of LIF neurons as well as due to recurrent synaptic connections. Recently, Bellec et al. [3] successfully trained RCNNs with slow temporal neuronal dynamics using a piecewise linear surroga”
A solution to the learning dilemma for recurrent networks of spiking neurons (2019, methodology): “We overcome this issue as in (Esser et al., 2016; Bellec et al., 2018): the non-existing derivative ∂z tj ∂v tj is replaced in simulations by a simple non-linear function of the membrane potential that is called the pseudo-derivative.”
Time2Vec: Learning a Vector Representation of Time (2019, background): “2) Event-MNIST: Sequential (event-based) MNIST is a common benchmark in sequence modeling literature (see, e.g. , [4, 6, 15]).”
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook (2021, methodology): “…Error Reassign-ment (SLAYER) algorithm [22], spatiotemporal backpropagation (STDB) [37], and a combination of BPTT with deep rewiring proposed in [36], which trains recurrent long short-term SNNs (LSNNs) with capabilities similar to conventional LSTMs. SLAYER was used to train a network for DVS…”
A critique of pure learning and what artificial neural networks can learn from animal brains (2019, background): “The idea that animals are predisposed to learn certain things rapidly is related to “ meta-learning ” and “ inductive biases ” in AI research and cognitive science 9,39 – 41 .”
DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization (2021, background): “There has been a signiﬁcant amount of success with recent supervised learning algorithms [14], [24], [35]–[37] that can be broadly classiﬁed as conversion algorithms [14], [28], [38] and spike-based backpropagation algorithms [18], [35].”
Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection (2019, background): “That is, SNNs utilize temporal aspects in information transmission as in biological neural systems (Mainen and Sejnowski 1995), thus providing sparse yet powerful computing ability (Mostafa et al. 2017; Bellec et al. 2018).”
Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation (2020, methodology): “…of the spiking neuron is handled by either approximating the spiking neuron model as continuous and differentiable (Huh & Sejnowski, 2018) or by deﬁning a surrogate gradient as a continuous approximation of the real gradient (Wu et al., 2018; Bellec et al., 2018; Neftci et al., 2019).”
Spike-driven Transformer (2023, background): “Many biological mechanisms are leveraged to inspire SNN’s neuron modeling [1, 30], learning rules [31, 32], etc. Existing studies have shown that SNNs are more suited for incorporating with brain mechanisms, e.g., long short-term memory [33, 34], attention [35, 27], etc.”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, background): “[2] proposed the adaptive threshold spiking neuron to enhance computing and learning capabilities of SNNs, which was improved by [61] with learnable time constants.”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, methodology): “6(b)] [73], [107], [156], [169], [170], i.”
Deep Learning With Spiking Neurons: Opportunities and Challenges (2018, background): “Recent work (e.g., Diehl et al., 2016; Bellec et al., 2018 ) have shown how standard recurrent network architectures such as long short-term memory networks (LSTMs, Hochreiter and Schmidhuber, 1997) can be ported into the spiking domain, whereas Neil et al. (2016b) have shown a way to process…”