Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

By Nathaniel D. Daw, Yael Niv, Peter S. Dayan. 2005

2,511 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Neuroscience

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “This shift may arise from a rational arbitration between learning systems to balance the trade-off between flexibility and speed (Daw et al., 2005; Keramati, Dezfouli, & Piray, 2011).”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Related work by (Daw et al., 2005) proposes a Bayesian uncertainty based model for arbitrating competition among important subsystems of the human brain.”
Adaptive Online Planning for Continual Lifelong Learning (2019, background): “When uncertain about a course of action, humans use an elongated model-based search to evaluate long-term trajectories, but fall back on habitual behaviors learned with model-free paradigms when they are certain of what to do [3, 8, 9, 16].”
Model-Free Episodic Control (2016, background): “Humans and animals utilise multiple learning, memory, and decision systems each best suited to different settings [5, 33].”
Rule abstraction, model-based choice, and cognitive reflection (2016, background): “The two-step task is a sequentially structured choice task that dissociates model-free and model-based strategies in reinforcement learning, each of which determines how actions are evaluated and selected based on previous experiences (Daw et al., 2005).”
The Neurobiology of Decision: Consensus and Controversy (2009, neutral): “These proposals usually associate the different systems with different regions of the frontal cortex and striatum (Balleine et al., 2009), and raise the additional question of how these multiple valuations interact or combine to control behavior (Daw et al., 2005).”
TRoPICALS: a computational embodied neuroscience model of compatibility effects. (2010, methodology): “Erlhagen and Bicho (2006) show, both theoretically and through examples, how dynamic neural elds can be used as a general design tool for building biologically plausible cognitive architectures for controlling autonomous robot.”
Generalization of value in reinforcement learning by humans (2012, background): “Still, it remains less precisely understood how other forms of learning, possibly involving distinct cognitive and neural systems, contribute to choice (Doya, 1999; Daw et al., 2005; Daw & Shohamy, 2008).”
Neural computations associated with goal-directed choice. (2010, background): “Current Opinion in Neurobiology 2010, 20:262–270 do much better since it is able to pick the optimal action in every trial [10].”
Building Machines that Learn and Think Like People (2018, background): “This shift may arise from a rational arbitration between learning systems to balance the trade-off between flexibility and speed (Daw et al., 2005; Keramati, Dezfouli, & Piray, 2011).”
Modulators of decision making (2008, background): “Uncertainty in prediction can also favor one learning framework over another—for example, model-free temporal difference learning and model-based plannin”
Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task (2015, methodology): “This has been interpreted as a shift to model-free RL [4].”