Learning in spiking neural networks by reinforcement of stochastic synaptic transmission.

By H. Sebastian Seung. 2003

376 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “…Kistler, Gerstner, & van Hemmen, 1997; Maass, 1996, 1997; Maex & Orban, 1996; Nessler, Pfeiffer, Buesing, & Maass, 2013; Rezende & Gerstner, 2014; Seung, 2003; Song, Miller, & Abbott, 2000; Stemmler, 1996; Stoop, Schindler, & Bunimovich, 2000; Tsodyks, Pawelzik, & Markram, 1998; Tsodyks, Skaggs,…”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “This suggests that the brain may have discovered mechanisms of credit assignment quite diﬀerent from those dreamt up by machine learning.”
Neural syntax: cell assemblies, synapsembles and readers (2010, positive): “…of neuronal patterns can be strengthened by reinforcers (i.e., goals), which optimize the connectivity of the upstream assembly post-hoc so that it will activate the reader more effectively on future occasions (Izhikevich, 2007; Legenstein and Maass, 2007; Maass et al., 2002; Seung, 2003).”
Copying and Evolution of Neuronal Topology (2008, methodology): “This can be abolished by assuming hard competition (winner-take-all dynamics) as is standard in many models [25,54].”
Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity (2007, methodology): “This problem was also approached using other reinforcement learning methods for generic spiking neural networks (Seung, 2003; Xie & Seung, 2004).”
Learning Spike-Based Population Codes by Reward and Population Feedback (2010, methodology): “But our population approach is not conﬁned to this model and could readily be adapted to use other reinforcement learning procedures at the single neuron level (Seung, 2003; Fiete & Seung, 2006; Florian, 2007).”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “Beyond neuromorphic applications, SNNs are also used to test theories about how natural intelligence may arise, from the higher level learning rules of the brain [55] and how memories are formed [56] down to the lower level dynamics at the neuronal and synaptic layers [57].”
A reward-modulated Hebbian learning rule can explain experimentally observed network reorganization in a brain control task (2010, background): “Either the tunable parameters of the system (weights) are perturbed (Jabri and Flower, 1992; Cauwenberghs, 1993; Seung, 2003) or the output of nodes in the network are perturbed (Mazzoni et al., 1991; Williams, 1992; Baxter and Bartlett, 2001; Fiete and Seung, 2006).”
Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment (2008, result): “Similar results could be obtained using a different learning algorithm, such as for example the biologically plausible algorithm proposed by Seung’s group [46,47], a kind of reinforcement learning.”
Random feedback weights support learning in deep neural networks (2014, background): “But reinforcement learning, which delivers the same reward signal to each neuron, is slow and scales poorly with network size 5,13,24 .”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, methodology): “Despite the methods that arise in reinforcement learning, like policy gradient [52, 28], temporal-difference learning [46, 14] and Q-learning [6], some heuristic phenomenological models based on STDP [15, 62] were proposed recently.”
Kickback Cuts Backprop's Red-Tape: Biologically Plausible Credit Assignment in Neural Networks (2014, methodology): “More recent approaches include REINFORCE (Williams 1992), the hedonistic neurons in (Seung 2003), and the neurons modeled using online learning in (Hu et al. 2013).”
Neuronal boost to evolutionary dynamics (2015, background): “We call attention to a partly related model by Seung [10] invoking ‘hedonistic synapses’ that would release neuro-transmitters stochastically, and an immediate reward would either strengthen or weaken them according to whether vesicle release of failure preceded reward, respectively.”
Reinforcement Learning, Spike-Time-Dependent Plasticity, and the BCM Rule (2007, background): “In Xie and Seung (2004) and Seung (2003), a more realistic neural model is used, but learning is based on spiking rates rather than on the membrane potential itself, and the derivation of the algorithm requires additional statistical assumptions about the spiking events.”
Theories of Error Back-Propagation in the Brain (2019, background): “The lack of local error representation has been addressed by early theories by proposing that the errors associated with individual neurons are not computed, but instead the synaptic plasticity is driven by a global error signal carried by neuromodulators [16 – 19].”
Decision making in recurrent neuronal circuits. (2008, background): “It remains to be seen how such a learning rule works when the outcome (reward or not) is revealed only long after the behavioral response, by incorporating either persistent neural activity that bridges the temporal gap between the two events or an ‘‘eligibility trace’’ in the synapses (Sutton and B”