Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity

By Razvan V. Florian. 2007

387 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

From modulated Hebbian plasticity to simple behavior learning through noise and weight saturation (2012, background): “Thus, while the dynamics of modulated Hebbian plasticity and modulated spike-timedependent plasticity (STDP) have been extensively investigated (Abbott, 1990; Montague et al., 1996; Florian, 2007; Porr and Wörgötter, 2007; Frémaux et al., 2010; Pfeiffer et al., 2010), the novelty of this work is the”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, methodology): “Furthermore, among all the flagged synapses only those that show, over many trials, a correlation with the reward signal will be consistently reinforced (Loewenstein and Seung, 2006; Legenstein et al., 2008).”
A reward-modulated Hebbian learning rule can explain experimentally observed network reorganization in a brain control task (2010, neutral): “…Mazzoni et al., 1991; Williams, 1992; Baxter and Bartlett, 1999; Loewenstein and Seung, 2006) and spiking-based (Xie and Seung, 2004; Fiete and Seung, 2006; Pfister et al., 2006; Baras and Meir, 2007; Farries and Fairhall, 2007; Florian, 2007; Izhikevich, 2007; Legenstein et al., 2008) models.”
Copying and Evolution of Neuronal Topology (2008, result): “The capacity of STDP to capture temporal relations consistent with causality rather than just correlations has been described by several authors [65,87,88,89].”
Learning Spike-Based Population Codes by Reward and Population Feedback (2010, methodology): “But our population approach is not confined to this model and could readily be adapted to use other reinforcement learning procedures at the single neuron level (Seung, 2003; Fiete and Seung, 2006; Florian, 2007).”
A Spiking Neural Model for Stable Reinforcement of Synapses Based on Multiple Distal Rewards (2013, methodology): “(2010) describe the R-Max plasticity rule (Xie & Seung, 2004; Pfister, Toyoizumi, Barber, & Gerstner, 2006; Florian, 2007), which is derived using the principle of reward maximization.”
Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules (2016, methodology): “The same argument of causality also applies to reward-modulated forms of STDP (Pfister et al., 2006; Florian, 2007).”
Introduction to spiking neural networks: Information processing, learning and applications. (2011, methodology): “It is worth noting that this learning rule can be used to modify synaptic connections both within feedforward (Florian 2007) and recurrent spiking networks (Florian 2005, Legenstein et al. 2008, Vasilaki et al. 2009).”
Rare Neural Correlations Implement Robotic Conditioning with Delayed Rewards and Disturbances (2013, background): “Izhikevich(2007)usestheprecisespike-timingtocre-atetracesaccordingtoatraditionalSTDPrule usestheprecisespike-timingtocre-atetracesaccordingtoatraditionalSTDPrule.”
Computational models of reinforcement learning: the role of dopamine as a reward signal (2010, background): “In most reward-modulated STDP models, an RL component updates the synaptic weights based on a reward signal, eligibility trace and learning curve (Florian 2007; Izhikevich 2007; Roberts et al. 2008; Thivierge et al. 2007).”
Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity (2010, methodology): “Learning rules like R-max (see Materials and Methods) that are derived from reward maximization principles (Xie and Seung, 2004; Pfister et al., 2006; Florian, 2007) use an eligibility trace without a bias ( eij 0), independent of the input statistics.”