Cognitively-Inspired Model for Incremental Learning Using a Few Examples

By Ali Ayub, Alan R. Wagner. 2020

51 citations recorded by the source corpus.

Incremental learning attempts to develop a classifier which learns continuously from a stream of data segregated into different classes. Deep learning approaches suffer from catastrophic forgetting when learning classes incrementally, while most incremental learning approaches require a large amount of training data per class. We examine the problem of incremental learning using only a few training examples, referred to as Few-Shot Incremental Learning (FSIL). To solve this problem, we propose a novel approach inspired by the concept learning model of the hippocampus and the neocortex that represents each image class as centroids and does not suffer from catastrophic forgetting. We evaluate our approach on three class-incremental learning benchmarks: Caltech-101, CUBS-200-2011 and CIFAR- 100 for incremental and few-shot incremental learning and show that our approach achieves state-of-the-art results in terms ofclassification accuracy over all learned classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Meta & Continual Learning, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

In Cognitive Science & Psychology

Citations in the corpus

Few-Shot Continual Active Learning by a Robot (2022, methodology): “As the FSCIL approaches are not designed for FoCAL, for a fair compairson we tested GBCL for the standard FSCIL setup [9] on the CIFAR-100 dataset [43].”
Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, methodology): “A new task that combines FSL and CIL is incremental few-shot learning (IFSL) [6, 44, 2].”
Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, background): “Tao et al. [39] propose a neural gas network to preserve the topology of the features in the base and new classes for the FSCIL task.”
EEC: Learning to Encode and Regenerate Images for Continual Learning (2021, methodology): “To avoid storing old class images, some approaches store features from the last fully-connected layer of the neural networks (Xiang et al., 2019; Hayes & Kanan, 2020; Ayub & Wagner, 2020b;d).”
Storing Encoded Episodes as Concepts for Continual Learning (2020, methodology): “…episodes Ny for a class y is calculated as (whole number): Ny(new) = Ny(1− Kr Kt−1 ) (4) To reduce the encoded episodes to Ny(new) for class y, inspired by the memory integration process, we use an incremental clustering process similar to Agg-Var clustering proposed in (Ayub & Wagner, 2019; 2020).”
What am I allowed to do here?: Online Learning of Context-Specific Norms by Pepper (2020, background): “Where, incremental learning is a field of machine learning focused on creating systems that can incrementally learn over time without requiring all the training data to be available in a single batch [4,6,12,20,5].”
Centroid Based Concept Learning for RGB-D Indoor Scene Classification (2019, methodology): “If the system does run out of memory, we can reduce the total number of centroids by applying Agg-Var clustering on the existing centroids with a larger distance threshold, thus combining similar centroids into a single centroid, decreasing the memory used [2].”
Learning Novel Objects Continually Through Curiosity (2021, methodology): “Agg-Var clustering [15], a cognitively-inspired approach, begins by creating one centroid from the first image in the training set of class y.”
Online Learning of Objects through Curiosity-Driven Active Learning (2021, methodology): “To overcome catastrophic forgetting, many different deep learning techniques have been proposed for continual learning [12], [13], [14], [15], [16], [17], [18].”
Don’t Forget to Buy Milk: Contextually Aware Grocery Reminder Household Robot (2022, methodology): “The implementation details and hyperparameters for the SUSTAIN network were used as in [32], [33].”
A Survey on Few-Shot Class-Incremental Learning (2023, background): “At present, there is no specific dataset for FSCIL, and most of them are made from existing datasets for new tasks.”
Tell me what this is: Few-Shot Incremental Object Learning by a Robot (2020, methodology): “An extension of [15] is presented in [18] for incremental object learning for a robot using a combination of a pre-trained feature extractor and NCM classifier which has been shown to be significantly inferior to other incremental learning approaches [3], [5].”
Online Continual Learning for Embedded Devices (2022, methodology): “Also relevant to this work are methods designed specifically for few-shot continual learning (Ayub & Wagner, 2020; 2021; Tao et al., 2020b;a).”