Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis.

By Michael J Frank, David Badre. 2012

329 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Motivation and cognitive control: from behavior to neural mechanism. (2015, background): “Model-free operation of control raises the amusing possibility that control function may, in some sense, be habitual (a description that fits some recent models of control function; e.g., Frank & Badre 2012).”
Hierarchical Active Inference: A Theory of Motivated Control (2018, background): “Calling on early cybernetic models [16–18], the view that the brain uses control hierarchies has inspired many recent proposals [19–25]; for example, hierarchical temporal structures [26], hierarchical reinforcement learning [9], hierarchical mixture of experts [23], distributed adaptive control [8,”
How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis (2012, background): “…in other work we have emphasized an analogous role of other BG circuits in the learning of when and when not to gate information into WM (Frank et al. , 2001; O’Reilly & Frank, 2006; Frank & Badre, 2011; Collins & Frank, unpublished data; see also Todd et al. , 2008; Gruber et al. , 2006).”
A computational model of inhibitory control in frontal cortex and basal ganglia. (2011, background): “…projections from both DLPFC to SEF and pre-SMA (Lu, Preston, & Strick, 1994; Y. Wang, Isoda, Matsuzaka, Shima, & Tanji, 2005) and to striatum to affect response gating (Doll, Jacobs, Sanfey, & Frank, 2009; Frank & Badre, 2012; Haber, 2003), and from SEF to FEF (Huerta, Krubitzer, & Kaas, 1987).”
Goals and Habits in the Brain (2013, unknown): “These could depend sensitively on the hierarchical architectures of cognitive control in lateral and medial prefrontal regions and their striatal connections (Frank and Badre, 2012; Koechlin and Hyafil, 2007; Koechlin et al., 2003).”
Cognitive Bias Modification and Cognitive Control Training in Addiction and Related Psychopathology (2013, background): “There is emerging evidence that this “motivational homunculus problem” can be solved by modeling the interplay between the development of motivation and control (Frank & Badre, 2012; Hazy, Frank, & O’Reilly, 2006, 2007).”
Model-based hierarchical reinforcement learning and human action control (2014, methodology): “In recent work, we and others have proposed that the human brain might leverage HRL-like representations and procedures in order to deal tractably with complex real-world decision problems [10,14,15].”
THE COGNITIVE NEUROSCIENCE OF WORKING MEMORY (2007, background): “Recently, Badre and Frank have provided computational and empirical evidence for the possible mechanisms underlying working memory gating (Badre & Frank, 2012, Frank and Badre 2012, Chatham et al., 2013).”
The Role of Executive Function in Shaping Reinforcement Learning. (2020, background): “There is recent computational, behavioral, and neural evidence that hierarchies of RL computations happen in parallel over more and more abstract types of states and choices, facilitating complex learning abilities [52, 53, 50].”
The necessity of rostrolateral prefrontal cortex for higher-level sequential behavior (2015, unknown): “Further, more recent conceptions of the rostro-caudal organization of lateral frontal cortex have come to emphasize working memory gating and the demand for multiple contingent gating responses arising from fronto-striatal interactions (Badre and Frank, 2012; Chatham et al., 2014; Frank and Badre, 2”