Gradient learning in spiking neural networks by dynamic perturbation of conductances.

By Ila Rani Fiete, H. Sebastian Seung. 2006

133 citations recorded by the source corpus.

We present a method of estimating the gradient of an objective function with respect to the synaptic weights of a spiking neural network. The method works by measuring the fluctuations in the objective function in response to dynamic perturbation of the membrane conductances of the neurons. It is compatible with recurrent networks of conductance-based model neurons with dynamic synapses. The method can be interpreted as a biologically plausible synaptic learning rule, if the dynamic perturbations are generated by a special class of "empiric" synapses driven by random spike trains from an external source.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

In Reinforcement Learning

Citations in the corpus

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation (2013, methodology): “Fiete and Seung (2006) end up proposing a gradient estimator that looks like a correlation between the reward and the perturbation , very similar to that presented in Section 3, and which is a special case of the REINFORCE (Williams, 1992) algorithm.”
Estimating or Propagating Gradients Through Stochastic Neurons (2013, methodology): “Fiete and Seung (2006) end up proposing a gradient estimator that looks like a correlation between the reward and the perturbation , just like what we obtain in Section 3.”
A reward-modulated Hebbian learning rule can explain experimentally observed network reorganization in a brain control task (2010, background): “However,whileithadto beassumedbyFieteandSeung(2006)thattheexperimenter signal[ (cid”
Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, methodology): “Recently, Miconi (2016) used a “node perturbation”-based (Fiete & Seung, 2006; Fiete et al., 2007; Hoerzer et al., 2014) algorithm with an error signal at the end of each trial to train RNNs for several cognitive tasks, and indeed, node perturbation is closely related to the REINFORCE algorithm…”
An objective function for STDP (2015, methodology): “Many approaches (Fiete and Seung, 2006; Rezende and Gerstner, 2014) rely on variants of the REINFORCE algorithm (Williams, 1992) to estimate the gradient of a global objective function (basically by correlating stochastic variations at each neuron with the changes in the global objective).”
Motor learning with unstable neural representations. (2007, background): “Recent studies suggest how reinforcement learning algorithms can be implemented in biophysically realistic, spiking neural circuits (Fiete and Seung, 2006; Seung, 2003; Xie and Seung, 2004).”
Learning Spike-Based Population Codes by Reward and Population Feedback (2010, methodology): “But our population approach is not confined to this model and could readily be adapted to use other reinforcement learning procedures at the single neuron level (Seung, 2003; Fiete and Seung, 2006; Florian, 2007).”
Reinforcement Learning on Slow Features of High-Dimensional Input Streams (2010, background): “Several theoretical studies have investigated biologically plausible reward-based learning rules [46–55].”