Reward-based training of recurrent neural networks for cognitive and value-based tasks

By H. Francis Song, Guangyu Robert Yang, Xiao-Jing Wang. 2016

176 citations recorded by the source corpus.

Trained neural network models, which exhibit many features observed in neural recordings from behaving animals and whose activity and connectivity can be fully analyzed, may provide insights into neural mechanisms. In contrast to commonly used methods for supervised learning from graded error signals, however, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when the optimal behavior depends on an animal’s internal judgment of confidence or subjective preferences. Here, we describe reward-based training of recurrent neural networks in which a value network guides learning by using the selected actions and activity of the policy network to predict future reward. We show that such models capture both behavioral and electrophysiological findings from well-known experimental paradigms. Our results provide a unified framework for investigating diverse cognitive and value-based computations, including a role for value representation that is essential for learning, but not executing, a task.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex (2020, background): “Research in reinforcement learning has helped provide some insight into how the PFC may transform experiences into operational schemas [6],[7],[8].”
Reinforcement Learning, Fast and Slow. (2019, background): “And indeed, researchers have started to take notice [7, 8].”
Artificial Neural Networks for Neuroscientists: A Primer (2020, background): “1052 Neuron 107, September 23, 2020 confidence about their choice (Kiani and Shadlen, 2009; Song et al., 2017).”
Deep neuroethology of a virtual rodent (2019, background): “While some efforts have been made to analyze neural activity in reduced systems trained to solve multiple tasks (Song et al., 2017; Yang et al., 2019), those studies lacked the important element of motor control in a physical environment.”