Learning from Sensory and Reward Prediction Errors during Motor Adaptation

By Jun Izawa, Reza Shadmehr. PLoS Comput. Biol., 2011

473 citations recorded by the source corpus.

Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.

Subfields: Cognitive Neuroscience, Social Psychology

View Paper: https://www.semanticscholar.org/paper/d48a32e01ed4119e4605a8674d8541e944d6ced2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Social Psychology

Citations in the corpus

A predictive network architecture for a robust and smooth robot docking behavior (2012, methodology): “This reinforcement learning algorithm is similar to motor adaptation according to error-prediction in biology [33, 34].”
Principles of sensorimotor learning (2011, background): “Although reinforcement learning in human motor tasks has not yet received much attention, recent evidence indicates that reaching movements and saccades can be changed through explicit rewards 66,67 .”