From modulated Hebbian plasticity to simple behavior learning through noise and weight saturation

By Andrea Soltoggio, Kenneth O. Stanley. 2012

33 citations recorded by the source corpus.

Synaptic plasticity is a major mechanism for adaptation, learning, and memory. Yet current models struggle to link local synaptic changes to the acquisition of behaviors. The aim of this paper is to demonstrate a computational relationship between local Hebbian plasticity and behavior learning by exploiting two traditionally unwanted features: neural noise and synaptic weight saturation. A modulation signal is employed to arbitrate the sign of plasticity: when the modulation is positive, the synaptic weights saturate to express exploitative behavior; when it is negative, the weights converge to average values, and neural noise reconfigures the network's functionality. This process is demonstrated through simulating neural dynamics in the autonomous emergence of fearful and aggressive navigating behaviors and in the solution to reward-based problems. The neural model learns, memorizes, and modifies different behaviors that lead to positive modulation in a variety of settings. The algorithm establishes a simple relationship between local plasticity and behavior learning by demonstrating the utility of noise and weight saturation. Moreover, it provides a new tool to simulate adaptive behavior, and contributes to bridging the gap between synaptic changes and behavior in neural computation.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Short-term plasticity as cause–effect hypothesis testing in distal reward learning (2014, background): “(Montague et al, 1995; Soltoggio and Stanley, 2012), fail unless reward is simultaneous with the stimuli.”
Learning the rules of a game: Neural conditioning in human-robot interaction with delayed rewards (2013, background): “neuromodulation [22], it drives plasticity to reinforce rewardrelated stimuli and actions.”
Rare Neural Correlations Implement Robotic Conditioning with Delayed Rewards and Disturbances (2013, methodology): “The increasing evidence of the important role of neuromodulation in reward-driven learning led to the formulation of models of modulated plasticity with rate-based neurons (e.g., Montague et al., 1996; Alexander and Sporns, 2002; Sporns and Alexander, 2002; Ziemke and Thieme, 2002; Soltoggio et al.,”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, background): “Another type of learner network includes neuromodulation , wherein the output of one neuron in a network can change the learning rates of other connections in the network [40, 187, 125, 161, 141, 160].”
Real-time Hebbian Learning from Autoencoder Features for Control Tasks (2014, background): “…artificial life and related fields (Baxter, 1992; Floreano and Urzelai, 2000; Niv et al., 2002; Soltoggio et al., 2008, 2007; Soltoggio and Jones, 2009; Soltoggio and Stanley, 2012; Risi and Stanley, 2010; Risi et al., 2011; Risi and Stanley, 2012; Stanley et al., 2003; Coleman and Blair, 2012).”