Neural heterogeneity promotes robust learning

By Nicolas Perez-Nieves, Vincent C. H. Leung, Pier Luigi Dragotti, Dan F. M. Goodman. Nature Communications, 2020

298 citations recorded by the source corpus.

The brain is a hugely diverse, heterogeneous structure. Whether or not heterogeneity at the neural level plays a functional role remains unclear, and has been relatively little explored in models which are often highly homogeneous. We compared the performance of spiking neural networks trained to carry out tasks of real-world difficulty, with varying degrees of heterogeneity, and found that heterogeneity substantially improved task performance. Learning with heterogeneity was more stable and robust, particularly for tasks with a rich temporal structure. In addition, the distribution of neuronal parameters in the trained networks is similar to those observed experimentally. We suggest that the heterogeneity observed in the brain may be more than just the byproduct of noisy processes, but rather may serve an active and important role in allowing animals to learn in changing environments. The authors show that heterogeneity in spiking neural networks improves accuracy and robustness of prediction for complex information processing tasks, results in optimal parameter distribution similar to experimental data and is metabolically efficient for learning tasks at varying timescales.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/16f9da2d7b95a55b699bc260341a66fd474a4c10

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “Beyond neuromorphic applications, SNNs are also used to test theories about how natural intelligence may arise, from the higher level learning rules of the brain [55] and how memories are formed [56] down to the lower level dynamics at the neuronal and synaptic layers [57].”