A solution to the learning dilemma for recurrent networks of spiking neurons

By Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert A. Legenstein, Wolfgang Maass. Nature Communications, 2019

631 citations recorded by the source corpus.

Recurrently connected networks of spiking neurons underlie the astounding information processing capabilities of the brain. Yet in spite of extensive research, how they can learn through synaptic plasticity to carry out complex network computations remains unclear. We argue that two pieces of this puzzle were provided by experimental data from neuroscience. A mathematical result tells us how these pieces need to be combined to enable biologically plausible online network learning through gradient descent, in particular deep reinforcement learning. This learning method–called e-prop–approaches the performance of backpropagation through time (BPTT), the best-known method for training recurrent neural networks in machine learning. In addition, it suggests a method for powerful on-chip learning in energy-efficient spike-based hardware for artificial intelligence. Bellec et al. present a mathematically founded approximation for gradient descent training of recurrent neural networks without backwards propagation in time. This enables biologically plausible training of spike-based neural network models with working memory and supports on-chip training of neuromorphic hardware.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/858549b00245aadc92f91a2540f01398f5f389ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Training Spiking Neural Networks Using Lessons From Deep Learning (2021, methodology): “Equation (12) describes how to propagate gradient-related terms forward in time to implement online learning, where such terms could be approximated by eligibility traces that keep track of presynaptic neuron activity in the form of calcium ions and fades over time [109], [234].”
Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (2024, background): “Some works introduce eligibility traces to improve BPTT for online training through time (Zenke & Ganguli, 2018; Bellec et al., 2020; Bohnstingl et al., 2022; Xiao et al., 2022), in which OTTT is an efficient and scalable approach with more theoretical guarantees.”
Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook (2021, background): “Longer term, more general algorithms, such as eligibility propagation [47] and equilibrium propagation [131] that approximate BPTT without the anticausal requirement of propagating information back in time, may succeed in scaling up online learning.”
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures (2019, methodology): “Recently, e-prop Bellec et al. (2019) presented an approximation method to bypass neuronal state savings for enhancing the computational eﬃciency of BPTT.”
The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks (2019, methodology): “Furthermore, there are biologically inspired online approximations of surrogate gradients [2], [73], [74] and, finally, a body of work has used biologically motivated spiketiming dependent plasticity (STDP)-like learning rules [75], [76] (see [4] for a comprehensive review).”