Goals and Habits in the Brain

By R. J. Dolan, Peter S. Dayan. 2013

938 citations recorded by the source corpus.

An enduring and richly elaborated dichotomy in cognitive neuroscience is that of reflective versus reflexive decision making and choice. Other literatures refer to the two ends of what is likely to be a spectrum with terms such as goal-directed versus habitual, model-based versus model-free or prospective versus retrospective. One of the most rigorous traditions of experimental work in the field started with studies in rodents and graduated via human versions and enrichments of those experiments to a current state in which new paradigms are probing and challenging the very heart of the distinction. We review four generations of work in this tradition and provide pointers to the forefront of the field’s fifth generation.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

Citations in the corpus

Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task (2015, background): “Humans and other animals are thought to use a mixture of different strategies to learn to choose actions that lead to positive outcomes and prevent negative outcomes [1,2].”
Replay Comes of Age. (2017, background): “Given our revised understanding of hippocampal replay, it is interesting to note that psychosis has been proposed to result from an overdominant internal model (Dolan & Dayan 2013).”
Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent (2019, background): “In the process of reinforcement learning, the agent’s attempt in each state was like the regulation process of dopamine in the brain (Dolan and Dayan, 2013).”
Model-based learning protects against forming habits (2015, background): “Moreover, individual differences in the propensity for model-based (over model-free) choice are positively correlated with gray matter volume in the vmPFC and caudate (Voon et al., 2014), regions that are critical for goal-directed control over action, in particular.”
A free energy principle for a particular physics (2019, background): “This likelihood is sometimes characterised as a state-action policy in psychology and machine learning; e.g., (Dolan and Dayan, 2013; Gershman and Daw, 2017; LeCun et al., 2015).”
Model-based hierarchical reinforcement learning and human action control (2014, background): “However, attention has subsequently expanded to consider the potential relevance of model-based RL [4,5].”
Cognitive Control Predicts Use of Model-based Reinforcement Learning (2014, background): “A number of theories across neuroscience, cognitive psychology, and economics posit that choices may arise from at least two distinct systems (Dolan & Dayan, 2013; Kahneman, 2011; Balleine & O ʼ Doherty, 2009; Daw, Niv, & Dayan, 2005; Loewenstein, 1996).”
Model-Based Reasoning in Humans Becomes Automatic with Training (2015, background): “This is suggested to rendermodel-based reasoning more flexible but at a heightened computational cost [3].”
Active inference and epistemic value (2015, background): “It is tempting to associate this (inevitable and emergent) progression from exploration to exploitation with the transformation of goal-directed behavior into habits (Balleine & Dickinson, 1998; Dolan & Dayan, 2013; Pezzulo, Rigoli, & Chersi, 2013).”
Disorders of compulsivity: a common bias towards learning habits (2014, neutral): “…these disorders are also associated with a significant shift in habit formation , evident early in the learning of a new decision problem, and that the abnormality can be quantified in terms of a detailed computational learning mechanism with strong neural foundations, model-free learning (3, 5).”
Formalizing Neurath’s Ship: Approximate Algorithms for Online Causal Learning (2016, background): “Such representations enable humans to exploit their physical and social environments in flexible and inherently model-based ways (Dolan & Dayan, 2013; Griffiths & Tenenbaum, 2007; Sloman, 2005).”
If deep learning is the answer, what is the question? (2020, background): “Stitching together multiple patterns of association, and learning their structures, could enable animals to learn a comprehensive model of the world that can be used for navigation, inference and planning 106 .”