Reinforcement learning in populations of spiking neurons

By Robert Urbanczik, Walter Senn. 2008

110 citations recorded by the source corpus.

Population coding is widely regarded as an important mechanism for achieving reliable behavioral responses despite neuronal variability. However, standard reinforcement learning slows down with increasing population size, as the global reward signal becomes less and less related to the performance of any single neuron. We found that learning speeds up with increasing population size if, in addition to global reward, feedback about the population response modulates synaptic plasticity.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Short-term plasticity as cause–effect hypothesis testing in distal reward learning (2014, methodology): “Feedback signals to output neurons are demonstrated to help learning also in Urbanczik and Senn (2009); Soltoggio et al (2013a).”
Random feedback weights support learning in deep neural networks (2014, methodology): “But reinforcement learning, which delivers the same reward signal to each neuron, is slow and scales poorly with network size 5,13,24 .”
Towards deep learning with segregated dendrites (2016, background): “Altogether, these artificial aspects of current deep learning solutions to credit assignment have rendered many scientists skeptical to the proposal that deep learning occurs in the real brain (Crick, 1989; Grossberg, 1987; Harris, 2008; Urbanczik and Senn, 2009).”
Learning Spike-Based Population Codes by Reward and Population Feedback (2010, background): “It has been previously shown that on average, single-neuron performance does not increase with population size and may in fact decrease slowly (Urbanczik & Senn, 2009).”
Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, unknown): “…the biological substrates of learning, in particular reward-dependent synaptic plasticity ( Seung, 2003 ; Soltani et al., 2006 ; Izhikevich, 2007 ; Urbanczik and Senn, 2009 ; Fre´maux et al., 2010 ; Soltani and Wang, 2010 ; Hoerzer et al., 2014 ; Brosch et al., 2015 ; Friedrich and Lengyel, 2016…”
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules (2016, methodology): “…at the junction leads to a reward or not (Arleo and Gerstner, 2000; Foster et al., 2000; Xie and Seung, 2004; Pfister et al., 2006; Baras and Meir, 2007; Florian, 2007; Di Castro et al., 2009; Sheynikhovich et al., 2009; Urbanczik and Senn, 2009; Vasilaki et al., 2009; Frémaux et al., 2010, 2013).”
Spatio-Temporal Credit Assignment in Neuronal Population Learning (2011, unknown): “Previous modeling of population learning has already considered the modulation of plasticity by feedback from the decision circuitry [16,34].”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, background): “, 2013; Rezende and Gerstner, 2014); as well as from additional factors that indicate whether a specific neuron in a population spikes in agreement with the majority of that population (Urbanczik and Senn, 2009).”
A Spiking Neural Model for Stable Reinforcement of Synapses Based on Multiple Distal Rewards (2013, background): “The concept of an attenuated reward signal has been considered before (see Fr´emaux et al., 2010; Urbanczik & Senn, 2009).”
Reinforcement Learning on Slow Features of High-Dimensional Input Streams (2010, background): “The learning rule for the network is then unchanged, each neuron tries to optimize the reward independently from the others [40], but see [41].”