Metalearning and neuromodulation

By Kenji Doya. 2002

705 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “Such a modulation of signals, or gated learning (Abbott, 1990), allows for the regulation and learning of behaviors as shown in numerous computational models (Baxter et al., 1999; Suri et al., 2001; Birmingham, 2001; Alexander and Sporns, 2002; Doya, 2002; Fujii et al., 2002; Suri, 2002; Ziemke and ”
Mapping Information Flow in Sensorimotor Networks (2006, background): “in the presence of changes in innately salient sensory stimulation, analogous to biological neuromodulatory systems (e.g., [”
Modulators of decision making (2008, background): “The network is affected by the sensory and contextual information represented in the cortex, as well as in diffuse neurochemical systems, such as serotonin, norepinephrine and acetylcholin”
Robot coverage control by evolved neuromodulation (2013, background): “The idea of relating neuromodulatory substances to properties and learning parameters has been explored [14], [22], but to the best of our knowledge has not previously been evolved.”
Distributional reinforcement learning in the brain (2020, background): “Neuromodulators such as serotonin and norepinephrine, acting in cortical or striatal areas, are good candidates for tuning such a gain mechanism [49,50].”
From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences (2011, background): “In this scheme, a phasic dopamine signal interpreted as TD-error plays a double role (Houk et al., 1995; Barto, 1995; Montague et al., 1996; Schultz et al., 1997; Suri and Schultz, 2001; Doya, 2002; Baldassarre, 2002; Khamassi et al., 2005).”
The Cyber Rodent Project: Exploration of Adaptive Mechanisms for Self-Preservation and Self-Reproduction (2005, methodology): “In applications of RL to real-world problems, the design of the reward function and the setting of meta-parameters, such as the discount factor, are crucial for successful achievement of the task (Doya, 2002).”
Emotions: from brain to robot. (2004, background): “Others have suggested that neuromodulation might be a key to meta-learning [57].”
From modulated Hebbian plasticity to simple behavior learning through noise and weight saturation (2012, background): “, 2010), making neuromodulation particularly suitable for modeling and implementing learning processes (Sporns and Alexander, 2002; Doya, 2002; Doya and Uchibe, 2005; Soula et al., 2005; Farries and Fairhall, 2007; Krichmar, 2008; Cox and Krichmar, 2009).”
Computational roles for dopamine in behavioural control (2004, background): “In one form of reinforcement learning called temporal-difference learning, a critical signal is the reward-prediction error (also called the temporal-difference, or TD error”
Intrinsically Motivated Learning in Natural and Artificial Systems (2013, background): “for another neuromodulator, serotonin, in aversive aspects of model-free control (Daw et al. 2002; Dayan and Huys 2009; Deakin and Graeff 1991; Doya 2002).”
Behavioral shifts and action valuation in the anterior cingulate cortex. (2008, neutral): “Specific ACC states and dedicated processing of outcomes in uncertain environments would convey the tuning of metaparameters such as learning/adaptation rate (Behrens et al., 2007; Doya, 2002).”
Habits, rituals, and the evaluative brain. (2008, background): “In line with this possibility, striosomes may represent state value in actor-critic architectures (Doya 2000, 2002) and thus may be critical components of striatum-based learning circuits.”
Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory (2011, background): “The ensuing phasic release of NE in cortical areas temporarily increases the responsivity (or gain) of these areas to their afferent input, selectiv ly potentiating the processing of these taskrelevant stimuli (Berridge & Waterhouse, 2003; Doya, 2002; Servan-Schreiber, Printz, & Cohen, 1990).”
Weighing up the Benefits of Work: Behavioral and Neural Analyses of Effort-Related Decision Making (2006, neutral): “…changes in dopaminergic VTA neurons could be encoding a reward prediction error signal that could be used by neural structures implementing reinforcement learning-type algorithms to guide reward seeking and choice (Doya, 2002; McClure, Daw, & Montague, 2003; Montague & Berns, 2002; Schultz, 2002 ).”