Multiple Model-Based Reinforcement Learning

By Kenji Doya, Kazuyuki Samejima, Ken-ichi Katagiri, Mitsuo Kawato. 2002

530 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computational Theory and Mathematics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computational Theory and Mathematics

In Reinforcement Learning

Citations in the corpus

Overcoming catastrophic forgetting in neural networks (2016, background): “…as supporting these capabilities by sustaining neural representations of task context that exert top-down gating inﬂuences on sensory processing, working memory, and action selection in lower-level regions [O’Reilly and Frank, 2006, Mante et al., 2013, Miller and Cohen, 2001, Doya et al., 2002].”
Deep learning in neural networks: An overview (2014, methodology): “Numerous alternative HRL techniques have been proposed (e.g., Ring, 1991, 1994; Jameson, 1991; Tenenberg et al., 1993; Moore and Atkeson, 1995; Precup et al., 1998; Dietterich, 2000b; Menache et al., 2002; Doya et al., 2002; Ghavamzadeh and Mahadevan, 2003; Barto and Mahadevan, 2003; Samejima et al.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “Doya et al. (2002) proposed a modular control architecture using multiple prediction models based on the computational model of the cerebellum proposed by Wolpert, Miall, and Kawato (1998).”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “Numerous alternative HRL techniques have been proposed [204, 206, 117, 279, 295, 171, 195, 50, 162, 51, 15, 215, 11, 260].”
Model-Based Reinforcement Learning with Value-Targeted Regression (2020, background): “, 1998) and recent years have witnessed phenomenal empirical advances such as in games, robotics and science Mnih et al. (2015); Silver et al.”
Computational models of reinforcement learning: the role of dopamine as a reward signal (2010, background): “3; Bertin et al. 2007; and for other application of the MMRL framework Doya et al. 2002; Samejima et al. 2003 ).”
Medial prefrontal cortex as an action-outcome predictor (2011, background): “Other reinforcement learning models have been developed with vector rather than scalar learning signal”
A unifying computational framework for motor control and social interaction. (2003, background): “Human neuroimaging and magnetic stimulation studies have also shown that the areas associated with action are also active during imitation and observation (Fadiga et al. 1995, 2002; Iacoboni et al. 1999; Grezes et al. 2001).”
Goals and Habits in the Brain (2013, unknown): “Further, there are suggestions that there are multiple model-based controllers, i.e., a mixture model (Doya et al., 2002), in which the selection between them can have model-based or potentially model-free components.”
Solving Hidden-Semi-Markov-Mode Markov Decision Problems (2014, background): “Planning in such non-stationary environments has already been studied in the MDP framework [8] and in the reinforcement learning framework [7, 9, 15].”
Credit Assignment in Multiple Goal Embodied Visuomotor Behavior (2010, background): “This is different from Doya et al. (2002) where all modules contribute weighted by how well each module is predicting the dynamics of the world, irrespective of value or overall outcome.”
Multi-agent quadrotor testbed control design: integral sliding mode vs. reinforcement learning (2005, background): “Much work has been done on continuous state-action space reinforcement learning methods [13], [14].”
Non-Stationary Markov Decision Processes a Worst-Case Approach using Model-Based Reinforcement Learning (2019, methodology): “Similarly, the non-stationary environments considered in the literature often evolve abruptly at discrete transition epochs (Dit-Yan et al., 1999; Hadoux, 2015; Hadoux et al., 2014; Doya et al., 2002; Da Silva et al., 2006; Choi et al., 2000; 2001; Campo et al., 1991; Wiering, 2001).”
Neuronal Reward and Decision Signals: From Theories to Data. (2015, background): “Model-based learning involves two separate processes, the acquisition and updating of the model, and the influence of the model on reinforcement learning, which likely occurs by influencing predictions of the outcome (139, 575).”
Model-based predictions for dopamine (2017, unknown): “…model, ‘model-based prediction errors’ must be computed for every aspect of the model in parallel — a multi-dimensional (i.e., vector) prediction error that signals not only that there is a mismatch between expectation and reality, but exactly what dimension of prediction was misaligned [34,57,58].”
Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values (2009, background): “One approach to this “curse of dimensionality” (Bellman, 1957) is to “divide and conquer” (Ghahramani and Wolpert, 1997; Doya et al., 2002), subdividing a complicated problem into simpler subproblems.”