Actor-critic models of the basal ganglia: new anatomical and computational perspectives

By Daphna Joel, Yael Niv, Eytan Ruppin. Neural Networks, 2002

510 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/871f07197135c5d09c5d65d90b02c7f0aa75c246

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Molecular Biology

Citations in the corpus

Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. (2013, methodology): “…is possible to exploit a similar mechanism to implement the inhibitor within the current model, also considering that the actor-critic model is often used to capture the reward prediction error signalled by DA and the acquisition of instrumental skills by BG (Houk et al., 1995; Joel et al., 2002).”
The Drosophila Mushroom Body: From Architecture to Algorithm in a Learning Circuit. (2020, methodology): “This biphasic memory profile was also observed when ethanol was used as reinforcement (Kaun et al. 2011).”
Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective (2008, background): “…architectures for RL have inspired new interpretations of functional divisions of labor within the basal ganglia and cerebral cortex (see Joel, Niv, and Ruppin (2002) for a review ), and RL-based accounts have been advanced to address issues as diverse as motor control (e.g., Miyamoto,…”
Event-related potential studies of outcome processing and feedback-guided learning (2012, methodology): “According to the reinforcement learning theory of the ERN/FRN (RL-theory) the human brain solves reinforcement learning problems by implementing an “actor-critic architecture” (Barto, 1995; Joel et al., 2002).”
Hierarchical models of behavior and prefrontal function. (2008, background): “Extensive evidence indicates that standard RL algorithms might be relevant to understanding learning mechanisms in the brain [60, 61 ].”
Decision theory, reinforcement learning, and the brain (2008, background): “…the rules of reinforcement learning (O’Doherty et al., 2003, 2004; Barto, 1995; Montague et al., 1996; Schultz et al., 1997; Suri and Schultz, 1998; Joel et al., 2002; Balleine et al., 2007; Daw and Doya, 2006; Haruno et al., 2004) in a way that ties together the at least equally extensive data…”
The Role of Executive Function in Shaping Reinforcement Learning. (2020, background): “com temporal difference learning [28,29] and actor-critic learning [30,31].”
The Hierarchical Organisation of Cortical and Basal-Ganglia Systems: A Computationally-Informed Review and Integrated Hypothesis (2013, unknown): “Basal ganglia, instead, have a micro-architecture that makes it ideal for selection (Redgrave et al., 1999) and for trial-and-error learning guided by dopamine learning signals (Joel et al., 2002).”
Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia (2006, background): “…number of these existing models (Schultz et al., 1995; Houk et al., 1995; Schultz et al., 1997; Suri et al., 2001; Contreras-Vidal & Schultz, 1999; Joel et al., 2002), thereby demonstrating that an emerging consensus view of basal ganglia learning can be applied to more complex cognitive…”
Whole-brain mapping of direct inputs to midbrain dopamine neurons. (2012, unknown): “For example, various computational models of reinforcement learning posit an important role for direct projections of striatal neurons to dopamine neurons in calculating reward prediction error (Doya, 1999; Houk et al., 1996; Joel et al., 2002; Suri, 2002 ).”
Decision theory, reinforcement learning, and the brain (2010, background): “This comes on top of a huge body of results on the involvement of dopamine and its striatal projection in appetitive learning and appetitively motivated choice behavior (for some recent highlights, see Costa, 2007; Hyman, Malenka, & Nestler, 2006; Joel et al., 2002; Wickens, Horvitz, Costa, & Killcr”
Metalearning and neuromodulation (2002, background): “Another critical issue in the reinforcement learning model of the basal ganglia is how to compute the ‘temporal diﬀerence’ of the value function V ( s ), represented in the striatal output (Joel et al., 2002).”