Neuromodulation Improves the Evolution of Forward Models

By Mohammad Sadegh Norouzzadeh, Jeff Clune. 2016

16 citations recorded by the source corpus.

Many animals predict the outcomes of their actions by internal models. Such "forward models'' enable animals to rapidly simulate many actions without performing them to choose an appropriate action. Robots would similarly benefit from forward models. However, such models must change over time to account for changes in the environment or body, such as injury. Thus, forward models must not only be accurate, but also adaptable. Neural networks can learn complex functions with high accuracy, hence they are suitable candidates to build forward models for robots. Generally, neural networks are static, which means once they pass the training phase, their weights remain unchanged and they thus cannot adapt themselves if something about the world or their body changes. Plastic neural networks change their connections over time via local learning rules (e.g. Hebbian rule) and can thus deal with unforeseen changes. A more complex, yet still biologically-inspired, technique is neuromodulation, which can change per-connection learning rates in different contexts. In this paper, we test the hypothesis that neuromodulation may improve the evolution of forward models because it can heighten learning after drastic changes such as injury. We compare forward models evolved with neuromodulation to those evolved with static neural networks and Hebbian plastic neural networks. The results show that neuromodulation produces forward models that can adapt to changes significantly better than the controls. Our findings suggest that neuromodulation is an effective tool for enabling robots (and artificial intelligence agents, more generally) to have more adaptable, effective forward models.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “In Norouzzadeh and Clune (2016), the authors demonstrated that neuromodulation produces forward models that can adapt to changes signiﬁcantly better than the controls.”
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence (2019, background): “Another type of learner network includes neuromodulation , wherein the output of one neuron in a network can change the learning rates of other connections in the network [40, 187, 125, 161, 141, 160].”