Short-term plasticity as cause–effect hypothesis testing in distal reward learning

By Andrea Soltoggio. 2014

31 citations recorded by the source corpus.

Asynchrony, overlaps, and delays in sensory–motor signals introduce ambiguity as to which stimuli, actions, and rewards are causally related. Only the repetition of reward episodes helps distinguish true cause–effect relationships from coincidental occurrences. In the model proposed here, a novel plasticity rule employs short- and long-term changes to evaluate hypotheses on cause–effect relationships. Transient weights represent hypotheses that are consolidated in long-term memory only when they consistently predict or cause future rewards. The main objective of the model is to preserve existing network topologies when learning with ambiguous information flows. Learning is also improved by biasing the exploration of the stimulus–response space toward actions that in the past occurred before rewards. The model indicates under which conditions beliefs can be consolidated in long-term memory, it suggests a solution to the plasticity–stability dilemma, and proposes an interpretation of the role of short-term plasticity.

Subfields: Cognitive Neuroscience, Social Psychology, Reinforcement Learning, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Social Psychology

In Reinforcement Learning

In Neuroscience

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “More recently, Soltoggio (2015) proposed the use of short- and long-term plasticity for consolidating new information on the basis of a cause-effect hypothesis testing when learning with delayed rewards.”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Metaplasticity and consolidation mechanisms were suggested to address this problem (Finnie and Nader, 2012; Soltoggio, 2015).”
Continual Learning via Local Module Composition (2021, methodology): “Continual learning methods typically address the problem of forgetting through parameter regularization [46, 70, 99], replay [89, 78, 3, 73, 54, 12, 97, 39, 81, 13] or dynamic architectures (and MoEs) [83, 87, 52, 85, 53, 41].”
Learngene: From Open-World to Your Learning Task (2021, methodology): “There are three mainstream methods in deep learning including regularization approaches [38, 29, 32, 72, 31], dynamic-architectures approaches [50, 69, 14, 49], and memory-replay approaches [57, 54, 40, 31, 30].”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, methodology): “Several families of recent CL approaches use this setting, including regularization meth-ods (Kirkpatrick et al., 2017), data replay (Soltoggio, 2015), and dynamic architectures (Rusu et al., 2016) (see (Lange et al., 2019) and (Parisi et al., 2019) for comprehensive overviews).”
Biological underpinnings for lifelong learning machines (2022, methodology): “Noisy, spurious correlations can be filtered out by a synaptic consolidation mechanism that extracts cause effects in input–output streams 199 .”
Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning (2020, methodology): “Several families of recent CL approaches use this setting, including regularization methods [36], data replay [73], and dynamic architectures [66] (see Lange et al.”
A Spiking Neural Model for Stable Reinforcement of Synapses Based on Multiple Distal Rewards (2013, background): “Neural Computation 25, 123–156 (2013) c © 2012 Massachusetts Institute of Technology”
Evolving Neuromodulatory Topologies for Plasticity in Video Game Playing (2016, background): “It has potential to be combined with other life-long learning methods such as hypothesis testing in distal reward learning [22], larger indirectly encoded networks [19], [16], overcoming deception in evolution of cognitive behaviors [12], and learning big collections of different behavior [3].”