Model-based learning protects against forming habits

By Claire M. Gillan, A. Ross Otto, Elizabeth A. Phelps, Nathaniel D. Daw. 2015

249 citations recorded by the source corpus.

Studies in humans and rodents have suggested that behavior can at times be “goal-directed”—that is, planned, and purposeful—and at times “habitual”—that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions. However, little is known about how habits and goal-directed control arise. Specifically, in the present study we sought to reveal the trial-by-trial dynamics of instrumental learning that would promote, and protect against, developing habits. In two complementary experiments with independent samples, participants completed a sequential decision task that dissociated two computational-learning mechanisms, model-based and model-free. We then tested for habits by devaluing one of the rewards that had reinforced behavior. In each case, we found that individual differences in model-based learning predicted the participants’ subsequent sensitivity to outcome devaluation, suggesting that an associative mechanism underlies a bias toward habit formation in healthy individuals.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

Citations in the corpus

Psychology of Habit. (2016, background): “The RL approach can account not only for the insensitivity of habits to changes in reward value but also for the chunking feature of habit automaticity.”
Habit formation and change (2018, background): “Deliberation keeps learning processes focused on goals, whereas habit formation arises when goals recede in salience and inﬂuence [48].”
The successor representation in human reinforcement learning (2016, background): “This can be understood in terms of a hybrid SR-MB account analogous to the MB-MF hybrids 13,21 considered previously (Figure 5, right panel).”