Stochastic variational learning in recurrent spiking networks

By Danilo Jimenez Rezende, Wulfram Gerstner. 2014

84 citations recorded by the source corpus.

The ability to learn and perform statistical inference with biologically plausible recurrent networks of spiking neurons is an important step toward understanding perception and reasoning. Here we derive and investigate a new learning rule for recurrent spiking networks with hidden neurons, combining principles from variational learning and reinforcement learning. Our network defines a generative model over spike train histories and the derived learning rule has the form of a local Spike Timing Dependent Plasticity rule modulated by global factors (neuromodulators) conveying information about “novelty” on a statistically rigorous ground. Simulations show that our model is able to learn both stationary and non-stationary patterns of spike trains. We also propose one experiment that could potentially be performed with animals in order to test the dynamics of the predicted novelty signal.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Low-dimensional dynamics for working memory and time encoding (2020, background): “Methods have been proposed for biologically plausible learning in recurrent networks (62, 63), but it should not be assumed that these methods scale to harder problems, as this scaling has proven difficult for biological approximations to backpropagation for feedforward networks (64).”
An objective function for STDP (2015, methodology): “Many approaches (Fiete and Seung, 2006; Rezende and Gerstner, 2014) rely on variants of the REINFORCE algorithm (Williams, 1992) to estimate the gradient of a global objective function (basically by correlating stochastic variations at each neuron with the changes in the global objective).”
Towards Biologically Plausible Deep Learning (2015, methodology): “update rule (Rezende and Gerstner, 2014), which however relied on the REINFORCE algorithm (Williams, 1992) rather than on inference to obtain credit assignment to internal neurons.”
Synaptic Consolidation: From Synapses to Behavioral Modeling (2015, background): “…et al., 2006; O’Donnell and Sejnowski, 2014), explicit modeling of neuromodulator signals in terms of novelty and surprise (Schmidhuber, 2006; Rezende and Gerstner, 2014), as well as tests of the model in a broader set of behavioral paradigms (Wang et al., 2010; Ramirez et al., 2013; Li et…”
Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks (2019, background): “However, the injected noise needed for smoothing out the effect of binary nonlinearities often poses a challenge for optimization [28].”
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks (2017, background): “The problem of sequence learning in recurrent neural networks has also been studied as a variational learning problem [34, 35] and by combining adaptive control theory with heterogeneous neurons [36].”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, background): “…that not all neurons react in the same way to a modulator as implemented by the notation hi(M 3rd) of Type 2 rules (Brea et al., 2013; Rezende and Gerstner, 2014); as well as from additional factors that indicate whether a specific neuron in a population spikes in agreement with the majority of…”
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules (2016, methodology): “Traditional studies of dopamine-modulated plasticity in cortico-striatal synapses have also relied on “rate-based” protocols, where no particular attention is paid to the relative timing of pre- and postsynaptic spikes (see Reynolds and Wickens, 2002 and Jay, 2003, for reviews).”