Rare Neural Correlations Implement Robotic Conditioning with Delayed Rewards and Disturbances

By Andrea Soltoggio, Andre Lemme, René Felix Reinhart, Jochen J. Steil. 2013

20 citations recorded by the source corpus.

Neural conditioning associates cues and actions with following rewards. The environments in which robots operate, however, are pervaded by a variety of disturbing stimuli and uncertain timing. In particular, variable reward delays make it difficult to reconstruct which previous actions are responsible for following rewards. Such an uncertainty is handled by biological neural networks, but represents a challenge for computational models, suggesting the lack of a satisfactory theory for robotic neural conditioning. The present study demonstrates the use of rare neural correlations in making correct associations between rewards and previous cues or actions. Rare correlations are functional in selecting sparse synapses to be eligible for later weight updates if a reward occurs. The repetition of this process singles out the associating and reward-triggering pathways, and thereby copes with distal rewards. The neural network displays macro-level classical and operant conditioning, which is demonstrated in an interactive real-life human-robot interaction. The proposed mechanism models realistic conditioning in humans and animals and implements similar behaviors in neuro-robotic platforms.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Self-organizing neural integration of pose-motion features for human action recognition (2015, background): “For instance, the visual detection of dangerous events for assistive robotics such as fall events (Parisi and Wermter, 2013, 2015), and the recognition of actions with learning robots in HRI scenarios (Soltoggio et al., 2013a,b; Barros et al., 2014; Parisi et al., 2014a,b).”
Real-time Hebbian Learning from Autoencoder Features for Control Tasks (2014, background): “In particular, the RAAHN can be viewed as a platform for later integrating more advanced and realistic Hebbian learning regimes, e.g. with distal rewards (Soltoggio et al., 2013) to demonstrate more convincingly their full potential.”
Learning the rules of a game: Neural conditioning in human-robot interaction with delayed rewards (2013, methodology): “The Rarely Correlating Hebbian Plasticity (RCHP)[17], [18] is a type of Hebbian plasticity that extracts only rare correlations from the neural activity.”
Value and reward based learning in neurorobots (2013, background): “Soltoggio and colleagues introduce the principle of rare correlations to resolve this issue (Soltosggio et al., 2013).”
RM-SORN: a reward-modulated self-organizing recurrent neural network (2015, methodology): “Beside the tasks from Izhikevich, the model was also successfully applied in robotics for classical and operant conditioning of the humanoid robot iCub (Soltoggio et al., 2013).”