Task Agnostic Meta-Learning for Few-Shot Learning

By Muhammad Abdullah Jamal, Guo-Jun Qi, Mubarak Shah. 2018

539 citations recorded by the source corpus.

Meta-learning approaches have been proposed to tackle the few-shot learning problem. Typically, a meta-learner is trained on a variety of tasks in the hopes of being generalizable to new tasks. However, the generalizability on new tasks of a meta-learner could be fragile when it is over-trained on existing tasks during meta-training phase. In other words, the initial model of a meta-learner could be too biased towards existing tasks to adapt to new tasks, especially when only very few examples are available to update the model. To avoid a biased meta-learner and improve its generalizability, we propose a novel paradigm of Task-Agnostic Meta-Learning (TAML) algorithms. Specifically, we present an entropy-based approach that meta-learns an unbiased initial model with the largest uncertainty over the output labels by preventing it from over-performing in classification tasks. Alternatively, a more general inequality-minimization TAML is presented for more ubiquitous scenarios by directly minimizing the inequality of initial losses beyond the classification tasks wherever a suitable loss can be defined. Experiments on benchmarked datasets demonstrate that the proposed approaches outperform compared meta-learning algorithms in both few-shot classification and reinforcement learning tasks.

Subfields: Artificial Intelligence, Cancer Research, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cancer Research

In Meta & Continual Learning

Citations in the corpus

Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? (2020, neutral): “74 TAML [21] 64-64-64-64 51.”
How to train your MAML (2018, background): “Difﬁculty in replicating the speciﬁc result has also been noted before in Jamal et al. (2018).”
Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima (2021, methodology): “Current few-shot methods mainly include optimization-based methods [12, 23, 32, 39, 45, 46, 55] and metric-based methods [13, 19, 44, 49, 52, 58, 57, 53].”
Few-Shot Incremental Learning with Continually Evolved Classifiers (2021, methodology): “Optimization-based methods aims to design efficient learning paradigm that enables fast network adaptation given limited data [8, 16, 28, 29, 37].”
iTAML: An Incremental Task-Agnostic Meta-learning Approach (2020, background): “[10] present a task-agnostic meta-learning approach applicable only to few-shot learning.”
Interventional Few-Shot Learning (2020, background): “Few-Shot Learning (FSL) — the task of training a model using very few samples — is nothing short of a panacea for any scenario that requires fast model adaptation to new tasks [64], such as minimizing the need for expensive trials in reinforcement learning [29] and saving computation resource for li”
Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods (2019, background): “Both aim to transfer the semantic knowledge or the knowledge of learning (e.g., meta-learning [3], [4]) from the source tasks to the target ones.”
A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “Meta learning is naturally suitable for FSL, and many studies have used meta-learning as a means of FSL, enabling the model to learn from a small number of new task samples (Elsken et al., 2020; Jamal and Qi, 2019; Ren et al., 2018).”
Adversarial Meta-Learning (2018, unknown): “[9] proposed a novel paradigm based on entropy that can meta-learn an unbiased initial model to improve the generalizability of the meta-learner, which can be effectively applied to new tasks with no bias.”
WCP: Worst-Case Perturbations for Semi-Supervised Deep Learning (2020, background): “This idea has been further extended to train semisupervised classifiers [36, 14, 1, 33, 26, 27, 25, 20, 28, 6, 17, 19] by temporally fusing networks through the training process.”
Mutual CRF-GNN for Few-shot Learning (2021, background): “(2) Memory network based methods [25, 34, 35] try to store knowledge from seen tasks and then generalize it to new tasks.”
Prompt, Generate, Then Cache: Cascade of Foundation Models Makes Strong Few-Shot Learners (2023, background): “Also, meta-learning is proposed to improve the few-shot adaptation ability of the models by finding a set of initialized parameters that can rapidly adapt to novel domains [12,21,38, 43].”
Prototype Mixture Models for Few-shot Semantic Segmentation (2020, unknown): “Existing methods can be broadly categorized as either: metric learning [22,23,17], meta-learning [24,25,26,27], or data argumentation.”
Meta-Learning without Memorization (2019, methodology): “…are adapted in MAML (Zintgraf et al., 2019), by compressing the task embedding (Lee et al., 2019), through data augmentation from a GAN (Zhang et al., 2018), by using an auxiliary objective on task gradients (Guiroy et al., 2019), and via an entropy regularization ob-jective (Jamal & Qi, 2019).”
GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task (2023, methodology): “Existing FSL methods are mainly optimization-based [5, 10, 16] and metric-based [6, 28, 29].”
A Survey of Optimization Methods From a Machine Learning Perspective (2019, methodology): “ing algorithm is proposed to enhance the generalizability of meta-learner towards a variety of tasks, which achieves outstanding performance on few-shot classiﬁcation and reinforcement learning tasks [179]. D. Optimization in Variational Inference In the machine learning community, there are many at”
Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition From a Domain Adaptation Perspective (2020, methodology): “Following the general meta-learning principle [15, 28, 33], they set aside a validation set to guide how to weigh the training examples by gradient descent.”
Meta-Learning Requires Meta-Augmentation (2020, methodology): “For overﬁtting in MAML in particular, regularization has been done by encouraging the model’s output to be uniform before the base learner updates the model [15], limiting the updateable parameters [40], or regularizing the gradient update based on cosine similarity or dropout [12, 35].”
DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers (2020, unknown): “Optimization-based methods [2, 5, 9–11, 18, 26, 31–33, 36, 38, 40–42, 44, 46, 47, 54, 55, 58, 64, 79] target at effectively adapting model parameters to new tasks in the low-shot regime.”