Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios

By Andrea Soltoggio, John A. Bullinaria, Claudio Mattiussi, Peter Dürr, Dario Floreano. 2008

154 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Learning to Continually Learn (2020, background): “There is prior work in machine learning that harnesses neuromodulation techniques [7, 16, 39, 40, 42], but in such work neuromodulation directly modulates learning rates, instead of the approach taken in this work of directly modulating activations and thus indirectly controlling learning.”
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities (2018, neutral): “One element of the experiment was that neural connections could change during an agent’s lifetime through neuromodulatory learning [75].”
Differentiable plasticity: training plastic neural networks with backpropagation (2018, background): “For example, neuromodulation (the control of moment-to-moment plasticity by network activity) has been shown to improve the performance of plastic neural networks designed by evolutionary algorithms (Soltoggio et al., 2008).”
Rare Neural Correlations Implement Robotic Conditioning with Delayed Rewards and Disturbances (2013, background): “A reward is generally conveyed by means of a modulatory signal (Montague et al., 1996; Farries and Fairhall, 2007; Florian,2007; Porr and Wörgötter,2007; Soltoggio et al., 2008; Pfeiffer et al., 2010).”
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity (2018, methodology): “Evolved networks with neuromodulated plasticity were shown to outperform both non-neuromodulated and non-plastic networks in various tasks (e.g. Soltoggio et al. 2008; Risi & Stanley 2012; see Soltoggio et al. 2017 for a review).”
Artificial Evolution of Plastic Neural Networks: A Few Key Concepts (2014, methodology): “For instance, one can imagine a robot in a Tmaze that must go to the end of the maze where a reward has been put [15], [17], [18].”
Emergent vision system of autonomous virtual characters (2016, methodology): “Soltoggio et al. proposed the use of a model of neuromodulation in T-Maze [28] and in bee foraging behavior [27].”
Evolution of Cartesian Genetic Programs for Development of Learning Neural Architecture (2011, methodology): “suggested using a heterosynaptic rule to evolve plastic ANNs in a simple dynamic, reward based scenario (T-maze environment; Soltoggio et al., 2008).”
Evolving Spiking Networks with Variable Resistive Memories (2014, methodology): “To demonstrate the ability of variable RM synapses to handle dynamic reward scenarios, an experiment was simulated in a T-maze (e.g., Blynel and Floreano, 2003) scenario, as used by Soltoggio et al. (2008) to investigate the adaptivity of plastic networks.”
A unified approach to evolving plasticity and neural geometry (2012, background): “The most important aspect of this domain is that it is not the traditional discrete grid-world T-Maze that is popular in this area [21] [25]; rather, the more realistic domain presented here requires the robot to develop both collision avoidance and the ability to learn during its lifetime in a…”
From modulated Hebbian plasticity to simple behavior learning through noise and weight saturation (2012, methodology): “However, this process can be reversed by changing the sign of Hebbian plasticity by means of a modulatory term η (Abbott, 1990; Porr and Wörgötter, 2007; Florian, 2007; Soltoggio et al., 2008; Pfeiffer et al., 2010) in ∆w ji = η · v j · v i .”
Indirectly Encoding Neural Plasticity as a Pattern of Local Rules (2010, background): “Traditional approaches to evolving adaptive ANNs with direct encodings also evolve the coefficients of Equation (3) but because of the limitations of direct encodings often only employ one such evolved rule throughout all ANN connections [3, 18].”
Evolution of Social Representation in Neural Networks (2013, background): “Evolution of learning ability is made possible using neuro- modulation (Soltoggio et al., 2008).”
Real-time Hebbian Learning from Autoencoder Features for Control Tasks (2014, background): “…and learning, neural plasticity has long captivated artificial life and related fields (Baxter, 1992; Floreano and Urzelai, 2000; Niv et al., 2002; Soltoggio et al., 2008, 2007; Soltoggio and Jones, 2009; Soltoggio and Stanley, 2012; Risi and Stanley, 2010; Risi et al., 2011; Risi and Stanley,…”
Evaluating Topological Models of Neuromodulation in Polyworld (2014, background): “Soltoggio et al. (2008) showed how neuromodulation of synaptic plasticity could improve performance on tasks involving distant rewards with sparse learning events.”