A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement

By J. Houk, Joel L. Davis, D. Beiser. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “learn to select information by reinforcement learning (Houk et al., 1995; Graybiel, 2005), the cerebellum to implement fast time-scale computations possibly acquired with supervised learning (Wolpert et al.”
WRITING MEMORIES WITH LIGHT-ADDRESSABLE REINFORCEMENT CIRCUITRY (2009, background): “A variety of neuromorphic circuits for computing such signals have been proposed (Houk et al., 1995; Montague et al., 1995; Schultz et al., 1997; Wörgötter and Porr, 2005), but these remain speculative until actual biological examples are solved.”
What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? (1999, background): “…architecture can be implemented in the network linking frontal cortical areas, such as the supplementary motor area, and the basal ganglia, with the striatal striosome compartment implementing the critic and the matrix compartment implementing the actor (Barto et al., 1995; Houk et al., 1995).”
Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy (2013, background): “(D ) BG may contribute to reinforcement learning processes, including, but not limited to reward-based learning, such that those processes or actions that result in reinforcements will be repeated (e.g., Houk et al., 1995; Berns and Sejnowski, 1998; Suri et al., 2001).”
Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. (2013, methodology): “The organisation into channels of the direct pathway (suitably modulated by the indirect and hyperdirect pathways), has led to wide agreement that the BG are well suited to select secondary perceptual representations, actions, and other more complex cognitive contents when animals face selection pro”
Neuronal Reward and Decision Signals: From Theories to Data. (2015, background): “The eligibility traces may consist of intracellular calcium changes (154, 637), formation of calmodulin-dependent protein kinase II (236), IP3 increase in dopamine neurons (203), and sustained neuronal activity in striatum and frontal cortex (522) reflecting previous events and choices in prefrontal”
Metalearning and neuromodulation (2002, background): “In this paper, our main focus is on the roles of neuromodulators within the circuit of basal ganglia, which have been suggested as the major locus of reinforcement learning (Houk et al., 1995; Montague et al., 1996; Doya, 2000a).”
Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making (2006, unknown): “…and ventral striatal BG regions can support the critic function by learning to associate stimuli with affective states and driving dopaminergic firing in the midbrain (O'Reilly, Frank, Hazy, and Watz (in press); see also Brown, Bullock, and Grossberg (1999) and Houk, Adams, and Barto (1995)).”
On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks (2013, background): “Such feed-back loops are often present in computational models of cortex-basal ganglia-thalamus-cortex loops for action selection [32–34] and are implicit in actor-critic models of reinforcement learning [31].”
A neurobehavioral model of affiliative bonding: implications for conceptualizing a human trait of affiliation. (2005, background): “DA cells, most numerously in the VTA, respond vigorously to and in proportion to the magnitude of both conditioned and unconditioned incentive stimuli and in anticipation of reward (Bowman et al. 1996; Henriksen & Giacchino 1993; Houk et al. 1995; Koob et al. 1993; Le Moal & Simon 1991; Mark et al. ”
Hippocampal replays under the scrutiny of reinforcement learning models. (2018, background): “It was proposed in the mid 1990s that the Pavlovian and instrumental learning capabilities of mammals could be explained by MF-RL algorithms (Barto 1995; Houk et al. 1995; Schultz et al. 1997).”
Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. (2004, background): “Several neural computational models emphasize the suitability of cortico-striatal networks for such learning [85–88].”