In Search of the Neural Circuits of Intrinsic Motivation

By Frederic Kaplan, Pierre-Yves Oudeyer. 2007

150 citations recorded by the source corpus.

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of children's developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans. We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

What is Intrinsic Motivation? A Typology of Computational Approaches (2007, background): “…in particular of the LPM model, for certain research debates in developmental psychology [e.g., language acquisition (Oudeyer and Kaplan, 2006), development of imitation (Kaplan and Oudeyer, 2007b)] and proposed some hypotheses for putative underlying neural circuits (Kaplan and Oudeyer, 2007a).”
Multi-Armed Bandits for Intelligent Tutoring Systems (2013, background): “Such models were shown to reproduce qualitative properties of spontaneous exploration and information seeking in humans [33, 41, 6].”
An intrinsic value system for developing multiple invariant representations with incremental slowness learning (2013, background): “Dopamine’s potential role in intrinsic motivation has been discussed before [19], [43], but not with respect to the formal theory of curiosity [55], which predicts that intrinsic reward should be proportional to compression progress.”
How Evolution May Work Through Curiosity-Driven Developmental Process (2016, background): “In these Playground Experiments, a quadruped “learning” robot (the learner) is placed on an infant play mat with a set of nearby objects and is joined by an “adult” robot (the teacher), see Figure 1 (A) (Oudeyer and Kaplan, 2006; Kaplan and Oudeyer, 2007b; Oudeyer et al., 2007).”
What are intrinsic motivations? A biological perspective (2011, background): “they might be related to tonic dopamine [48].”
Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies. (2016, background): “Figure 3 (B) outlines a computational architecture, called R-IAC (Oudeyer, Kaplan et al. 2007; Moulin-Frier et al., 2014).”
From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences (2011, methodology): “(Kaplan and Oudeyer, 2007a)), developmental psychology (Can we reinterpret the developmental sequences of young children as progress niches ?”