Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules

By Nicolas Frémaux, Wulfram Gerstner. 2016

475 citations recorded by the source corpus.

Classical Hebbian learning puts the emphasis on joint pre- and postsynaptic activity, but neglects the potential role of neuromodulators. Since neuromodulators convey information about novelty or reward, the influence of neuromodulators on synaptic plasticity is useful not just for action learning in classical conditioning, but also to decide “when” to create new memories in response to a flow of sensory stimuli. In this review, we focus on timing requirements for pre- and postsynaptic activity in conjunction with one or several phasic neuromodulatory signals. While the emphasis of the text is on conceptual models and mathematical theories, we also discuss some experimental evidence for neuromodulation of Spike-Timing-Dependent Plasticity. We highlight the importance of synaptic mechanisms in bridging the temporal gap between sensory stimulation and neuromodulatory signals, and develop a framework for a class of neo-Hebbian three-factor learning rules that depend on presynaptic activity, postsynaptic variables as well as the influence of neuromodulators.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

The temporal paradox of Hebbian learning and homeostatic plasticity (2017, background): “While a detailed account of the role of neuromodulators goes beyond the scope of this article (but see [84,85] for reviews), here we note that”
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules (2018, unknown): “ works if each neuromodulator codes for a different combination of variables such as surprise, novelty or reward, just as we can use different coordinate systems to describe the same physical system (Fremaux and Gerstner, 2016). Thus, whether´ dopamine is purely reward related or also novelty relate”
Motor primitives in space and time via targeted gain modulation in cortical networks (2018, unknown): “The neuronal dynamics oscillate transiently, with a substantially lower frequency than either the fast variants or the slow variants trained by our reward-based learning rule.”
A solution to the learning dilemma for recurrent networks of spiking neurons (2019, background): “(5) Previous 3-factor learning rules for RL were usually of the form ∆W t = ηδēji [24, 8].”
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks (2020, background): “Reward-based learning of SNNs Reward-based learning of SNNs mimics the way the human brain learns by taking advantage of the reward or punishment signals induced by dopaminergic, serotonergic, cholinergic, or adrenergic neurons [13, 6, 41].”
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity (2018, background): “…Hebbian plasticity (e.g. the inherent interference between the unsupervised component of Hebbian learning and reward-based modiﬁcations; Frémaux et al. 2010; Frémaux & Gerstner 2015), so as to facilitate the automatic design of efﬁcient, self-contained reinforcement learning systems.”
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks (2017, background): “Before we move on to testing this learning rule, we first state a few of its noteworthy properties: (i) it has a Hebbian term which combines pre- and postsynaptic activity in a multiplicative manner, (ii) the learning rule is voltage-based, (iii) it is a nonlinear Hebbian rule due to the occurrence ”