Cognitive control over learning: creating, clustering, and generalizing task-set structure.

By A. G. Collins, Michael J Frank. 2013

389 citations recorded by the source corpus.

Executive functions and learning share common neural substrates essential for their expression, notably in prefrontal cortex and basal ganglia. Understanding how they interact requires studying how cognitive control facilitates learning, but also how learning provides the (potentially hidden) structure, such as abstract rules or task-sets, needed for cognitive control. We investigate this question from three complementary angles. First, we develop a new computational “C-TS” (context-task-set) model inspired by non-parametric Bayesian methods, specifying how the learner might infer hidden structure and decide whether to re-use that structure in new situations, or to create new structure. Second, we develop a neurobiologically explicit model to assess potential mechanisms of such interactive structured learning in multiple circuits linking frontal cortex and basal ganglia. We systematically explore the link betweens these levels of modeling across multiple task demands. We find that the network provides an approximate implementation of high level C-TS computations, where manipulations of specific neural mechanisms are well captured by variations in distinct C-TS parameters. Third, this synergism across models yields strong predictions about the nature of human optimal and suboptimal choices and response times during learning. In particular, the models suggest that participants spontaneously build task-set structure into a learning problem when not cued to do so, which predicts positive and negative transfer in subsequent generalization tests. We provide evidence for these predictions in two experiments and show that the C-TS model provides a good quantitative fit to human sequences of choices in this task. These findings implicate a strong tendency to interactively engage cognitive control and learning, resulting in structured abstract representations that afford generalization opportunities, and thus potentially long-term rather than short-term optimality.

Subfields: Cognitive Neuroscience, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Cognitive Science & Psychology

Citations in the corpus

Comparing continual task learning in minds and machines (2018, background): “Indeed, studies of expert knowledge (39) and laboratory-based categorization (40, 41) show that the task context is an excessively salient cue for retrieval, as if participants partitioned their knowledge between tasks even when it was unnecessary or detrimental to do so (42).”
Reconciling meta-learning and continual learning with online mixtures of tasks (2018, background): “In contrast, in order to account for this degree of task heterogeneity, people detect and adapt to novel contexts by attending to relationships between tasks [10].”
How Does Metacognition Contribute to the Regulation of Learning? An Integrative Approach (2014, background): “…cognitive control (e.g., gaining or losing control depending on contingent conditions, Skinner, 1996; forming of abstract rules or task sets, Collins & Frank, 2013), increase of time, effort, and attention on the task; initiation and termination of processing (Nelson & Narens, 1994) and…”
Online gradient-based mixtures for transfer modulation in meta-learning (2018, background): “In contrast, in order to deal with this degree of task heterogeneity, extensive task-switching literature reveals that people detect and readily adapt even in the face of significantly novel contexts (see Collins and Frank, 2013, for a review).”
Affective Determinants of Health Behavior (2018, background): “Importantly, recent computational models have demonstrated how symbolic processing, characteristic of reflective processing, can arise via dynamic interactions among multiple “loops” connecting the prefrontal cortex and basal ganglia (Collins & Frank, 2013; Kriete, Noelle, Cohen, & O’Reilly, 2013).”
If deep learning is the answer, what is the question? (2020, background): “Other evidence from human category learning implies that human knowledge may be actively partitioned by time and context 127,128 .”
The Role of Executive Function in Shaping Reinforcement Learning. (2020, background): “Indeed, humans appear to cluster subsets of actions with associated sensory contexts during instrumental learning [48, 49], and they do so in a manner which suggests that high-level inferences about task structure shape low-level reinforcement learning computations over actions.”
Cognitive Control Predicts Use of Model-based Reinforcement Learning (2014, methodology): “Recent research (Collins & Frank, 2013; Collins & Koechlin, 2012; Gershman & Niv, 2010) aims to extend principles of RL, hierarchically, to learning at this level.”
Computational psychiatry as a bridge from neuroscience to clinical applications (2016, neutral): “An alternative to a competitive account between MB and MF is a more integrated one where the goals driving MB evaluations are provided by MF processes 123 , for instance by more abstract plans in anterior CSTC circuits being reinforced by dopaminergic signals 124 .”
The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers (2014, methodology): “Alongside their Bayesian model, Collins and Frank (2013) presented a neural network model, centering on the gating mechanism discussed earlier. They showed that introducing a layered or hierarchical arrangement into the gating model (in much the same spirit as Reynolds & O’Reilly, 2009) can spontane”