Neural Plasticity and Minimal Topologies for Reward-Based Learning

By Andrea Soltoggio. 2008

16 citations recorded by the source corpus.

Artificial neural networks for online learning problems are often implemented with synaptic plasticity to achieve adaptive behaviour. A common problem is that the overall learning dynamics are emergent properties strongly dependent on the correct combination of neural architectures, plasticity rules and environmental features. Which complexity in architectures and learning rules is required to match specific control and learning problems is not clear. Here a set of homosynaptic plasticity rules is applied to topologically unconstrained neural controllers while operating and evolving in dynamic reward-based scenarios. Performances are monitored on simulations of bee foraging problems and T-maze navigation. Varying reward locations compel the neural controllers to adapt their foraging strategies over time, fostering online reward-based learning. In contrast to previous studies, the results here indicate that reward-based learning in complex dynamic scenarios can be achieved with basic plasticity rules and minimal topologies.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “In another study, Soltoggio (2008c) suggested that evolved modulatory neurons may be essential to separate the learning circuity from the input-output controller, effectively contributing to shortening the input-output pathways and speeding up decision processes.”
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks (2017, methodology): “Like real brains, information in ANNs is encoded in connection weights and patterns and learning involves the changing of those weights [10–13].”
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills (2015, background): “Hebb’s rule [45]) or infrequent reward signals [8, 46, 47].”
Evolving Spiking Networks with Variable Resistive Memories (2014, background): “Integration of neuroevolution with heterogeneous neuromoduation rules is investigated by Soltoggio (2008), and has been extended to robot controllers (Durr et al., 2008).”
Evolution of Plastic Learning in Spiking Networks via Memristive Connections (2012, background): “, [37]).”
Towards evolving spiking networks with memristive synapses (2011, methodology): “Integration of neuroevolution with heterogeneous neuromoduation rules (which are similar to the different memristor types used here) is investigated by Soltoggio (e.g., [ 27 ]) for agent navigation tasks in dynamic environments; results demonstrate that neuromodulation allows for the evolution of hi”
Indirectly Encoding Neural Plasticity as a Pattern of Local Rules (2010, background): “Traditional approaches to evolving adaptive ANNs with direct encodings also evolve the coefficients of Equation (3) but because of the limitations of direct encodings often only employ one such evolved rule throughout all ANN connections [3, 18].”