Model-Based Reasoning in Humans Becomes Automatic with Training

By Marcos Economides, Zeb Kurth-Nelson, Annika Lübbert, Marc Guitart-Masip, R. J. Dolan. 2015

80 citations recorded by the source corpus.

Model-based and model-free reinforcement learning (RL) have been suggested as algorithmic realizations of goal-directed and habitual action strategies. Model-based RL is more flexible than model-free but requires sophisticated calculations using a learnt model of the world. This has led model-based RL to be identified with slow, deliberative processing, and model-free RL with fast, automatic processing. In support of this distinction, it has recently been shown that model-based reasoning is impaired by placing subjects under cognitive load—a hallmark of non-automaticity. Here, using the same task, we show that cognitive load does not impair model-based reasoning if subjects receive prior training on the task. This finding is replicated across two studies and a variety of analysis methods. Thus, task familiarity permits use of model-based reasoning in parallel with other cognitive demands. The ability to deploy model-based reasoning in an automatic, parallelizable fashion has widespread theoretical implications, particularly for the learning and execution of complex behaviors. It also suggests a range of important failure modes in psychiatric disorders.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, General Decision Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In General Decision Sciences

Citations in the corpus

Learning to reinforcement learn (2016, methodology): “We considered four different models encompassing different combinations of three parameters: learning rate α , softmax inverse temperature β , and a lapse rate (cid:15) to account for unexplained choice variance not related to estimated value Economides et al. (2015).”
Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task (2015, background): “Indeed, it is only after substantial additional training [42] that apparently model-based human two-step behaviour becomes resistant to inference from cognitive load from a demanding secondary task performed in parallel [11].”
When Does Model-Based Control Pay Off? (2016, methodology): “This task has proven to be a useful and popular tool to characterize the neural [8, 10 – 18], behavioral [19 – 31] and clinical [32 – 35] implications of dual-process models within the reinforcement learning framework.”