Adaptive Critics and the Basal Ganglia

By J. Houk, Joel L. Davis, D. Beiser. 1994

Subfields: Reinforcement Learning, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Neuroscience

Citations in the corpus

E-Z Reader: A cognitive-control, serial-attention model of eye-movement behavior during reading (2006, background): “To address this issue , Reichle and Laurent ( 2005 ) examined the eye - movement behavior of an adaptive reading ' ' agent ' ' that was given the task of learning how to move its eyes ( via a reinforcement - learning algo - rithm ; Barto , 1995 ) so as to read as efficiently as possi - ble .”
Divide and Conquer: Hierarchical Reinforcement Learning and Task Decomposition in Humans (2013, background): “The most critical parallel pertains to midbrain dopaminergic function, which has been proposed to transmit signals comparable to the reward-prediction errors that lie at the heart of RL (Barto, 1995; Montague et al., 1996; Schultz et al., 1997).”
The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. (2002, methodology): “…our theory is firmly grounded in previous computational work associating phasic dopaminergic activity with the method of temporal differences (Barto, 1995; Berns et al., 2001; Braver & Cohen, 2000; Friston et al., 1994; Houk et al., 1995; Montague et al., 1996; O’Reilly et al., 2002; Schultz…”
Interactions between frontal cortex and basal ganglia in working memory: A computational model (2001, background): “This dovetails nicely with the widely acknowledged role that the basal ganglia, and the neuromodulator dopamine, play in reinforcement learning (e.g., Barto, 1995; Houk, Adams, & Barto, 1995; Schultz, Dayan, & Montague, 1997; Schultz, Romo, et al., 1995).”
The correlative triad among aging, dopamine, and cognition: current status and future prospects. (2006, background): “Traditionally, the caudate circuits are thought to be particularly relevant to complex cognitive functioning, whereas the putaminal circuits are primarily implicated in motor activity (Alexander et al., 1986; Bhatia and Marsden, 1994; Houk et al., 1995).”
What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? (1999, background): “Traditionally, the central executive was supposed to undertake the task of activating and connecting appropriate processing modules (Baddeley, 1992).”
Hierarchically organized behavior and its neural foundations: A reinforcement-learning perspective (2008, background): “…extended to research on midbrain dopaminergic function, where the temporal-difference learning paradigm provided a framework for interpreting temporal profiles of dopaminergic activity (Barto, 1995; Houk, Adams, & Barto, 1995; Montague, Dayan, & Sejnowski, 1996; Schultz, Dayan, & Montague, 1997).”
Tonic dopamine: opportunity costs and the control of response vigor (2007, unknown): “Computational models showed that this signal can be used efficiently both for learning to predict rewards and for learning to choose actions so as to maximize reward intake (Sutton and Barto 1990; Friston et al. 1994; Barto 1995; Montague et al. 1996; Schultz et al. 1997).”
Perceptions of moral character modulate the neural systems of reward during the trust game (2005, background): “This has led to the hypothesis that the caudate nucleus may serve as a key component of an 'actor-critic' model processing the contingent behavior that led to the feedback, with the purpose of guiding future action”
Getting formal with dopamine and reward. (2002, unknown): “What are the behavioral relationships of these opiate receptive neu-frontal cortex (Barto, 1995; Houk et al., 1995), and the characteristics of its teaching signal correspond in the rons in the first place?”