Comparing continual task learning in minds and machines

By Timo Flesch, Jan Balaguer, Ronald B. Dekker, Hamed Nili, Christopher Summerfield. 2018

156 citations recorded by the source corpus.

Significance Humans learn to perform many different tasks over the lifespan, such as speaking both French and Spanish. The brain has to represent task information without mutual interference. In machine learning, this “continual learning” is a major unsolved challenge. Here, we studied the patterns of errors made by humans and state-of-the-art neural networks while they learned new tasks from scratch and without instruction. Humans, but not machines, seem to benefit from training regimes that blocked one task at a time, especially when they had a prior bias to represent stimuli in a way that encouraged task separation. Machines trained to exhibit the same prior bias suffered less interference between tasks, suggesting new avenues for solving continual learning in artificial systems. Humans can learn to perform multiple tasks in succession over the lifespan (“continual” learning), whereas current machine learning systems fail. Here, we investigated the cognitive mechanisms that permit successful continual learning in humans and harnessed our behavioral findings for neural network design. Humans categorized naturalistic images of trees according to one of two orthogonal task rules that were learned by trial and error. Training regimes that focused on individual rules for prolonged periods (blocked training) improved human performance on a later test involving randomly interleaved rules, compared with control regimes that trained in an interleaved fashion. Analysis of human error patterns suggested that blocked training encouraged humans to form “factorized” representation that optimally segregated the tasks, especially for those individuals with a strong prior bias to represent the stimulus space in a well-structured way. By contrast, standard supervised deep neural networks trained on the same tasks suffered catastrophic forgetting under blocked training, due to representational interference in the deeper layers. However, augmenting deep networks with an unsupervised generative model that allowed it to first learn a good embedding of the stimulus space (similar to that observed in humans) reduced catastrophic forgetting under blocked training. Building artificial agents that first learn a model of the world may be one promising route to solving continual task performance in artificial intelligence research.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computer Vision and Pattern Recognition, Meta & Continual Learning, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

In Cognitive Science & Psychology

Citations in the corpus

Synaptic weight decay with selective consolidation enables fast learning without catastrophic forgetting (2019, background): “This is indeed a well known phenomenon in psychophysics, called roving (Tartaglia et al., 2009b; Parkosadze et al., 2008) or interleaved learning (Flesch et al., 2018).”
Orthogonal representations for robust context-dependent task performance in brains and neural networks (2022, positive): “In a previous study (Flesch et al., 2018), we quantified behavioral response patterns in this trees task by fitting a model that made choices according to the two orthogonal ground truth boundaries (Flesch et al.”
Representational Continuity for Unsupervised Continual Learning (2021, background): “Recently continual learning (Thrun, 1995) has gained a lot of attention in the deep learning community due to its ability to continually learn on a sequence of non-stationary tasks (Kumar & Daume III, 2012; Li & Hoiem, 2016) and close proximity to the human learning process (Flesch et al., 2018).”
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “In neuroscience, much work has been done to understand the brain’s ability to consolidate learning from earlier tasks, thereby making it relatively robust to forgetting (Flesch et al., 2018; Cichon & Gan, 2015; Yang et al., 2014).”
Bayesian Structure Adaptation for Continual Learning (2019, background): “Also, as a skill innate to humans (Flesch et al., 2018), it is naturally an interesting scientiﬁc problem to reproduce the same capability in artiﬁcial predictive modelling systems.”
Cognitively-Inspired Model for Incremental Learning Using a Few Examples (2020, background): “ only grow slowly with respect to memory and computational requirements as the number of training classes increase. Incremental learning may offer a number of important advantages over batch learning [12]. Speciﬁcally, because the classes and data used to train the model do not need to be determined”
Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation (2022, background): “Interleaved training was proposed as a solution for catastrophic forgetting [4,10,11].”
Brain-Inspired Model for Incremental Learning Using a Few Examples (2020, background): “[12] demonstrate that incremental learning improves human learning by fostering the development of higher order representations that promote task segregation.”
Partitioned Variational Inference: A unified framework encompassing federated and continual learning (2018, methodology): “…is enjoying a resurgence of interest ranging from deepening understanding of transfer learning and catastrophic forgetting [Goodfellow et al., 2014, Flesch et al., 2018], to developing learning algorithms for various models and applications [Broderick et al., 2013, Li and Hoiem, 2016, Kirkpatrick…”
Continual Learning Of Predictive Models In Video Sequences Via Variational Autoencoders (2020, background): “Several articles have tried to include continual learning (also known as lifelong learning) capabilities into artificial systems inspired by findings in psychology and neuroscience [7, 8].”
If deep learning is the answer, what is the question? (2020, background): “…who have been trained in a blocked regimen to classify naturalistic stimuli (trees) according to two orthogonal boundaries (their ‘leafiness’ versus their ‘branchiness’) perform better on a later interleaved test compared with those who experienced the same conditions at training and test 8 .”
Rethinking the Representational Continuity: Towards Unsupervised Continual Learning (2021, background): “Recently continual learning (Thrun, 1995) has gained a lot of attention in the deep learning community due to its ability to continually learn on a sequence of non-stationary tasks (Kumar & Daume III, 2012; Li & Hoiem, 2016) and close proximity to the human learning process (Flesch et al., 2018).”