Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks

By Haebeom Lee, Hayeon Lee, Donghyun Na, Saehoon Kim, Minseop Park, Eunho Yang, Sung Ju Hwang. 2019

116 citations recorded by the source corpus.

While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed. Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks, even when the number of instances per task and class largely varies. Moreover, they do not consider distributional difference in unseen tasks, on which the meta-knowledge may have less usefulness depending on the task relatedness. To overcome these limitations, we propose a novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning within each task. Through the learning of the balancing variables, we can decide whether to obtain a solution by relying on the meta-knowledge or task-specific learning. We formulate this objective into a Bayesian inference framework and tackle it using variational inference. We validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on multiple realistic task- and class-imbalanced datasets, on which it significantly outperforms existing meta-learning approaches. Further ablation study confirms the effectiveness of each balancing component and the Bayesian learning framework.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Meta-Learning in Neural Networks: A Survey (2020, background): “Meanwhile some hybrid approaches [100], [101], [111], [165] implement semiamortized learning by drawing on both feed-forward and optimization-based meta-learning in a single framework.”
Meta-Learning with Neural Tangent Kernels (2021, unknown): “In this setting, the state of arts are achieved by Bayesian TAML (Lee et al., 2020).”
SMIL: Multimodal Learning with Severely Missing Modality (2021, methodology): “Then, inspired by (Finn, Xu, and Levine 2018; Gordon et al. 2019; Lee et al. 2020a), we define the generative process as optimizing the likelihood in a meta-learning framework:”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “[291] a new model called BTAML for dealing with imbalanced data and also detection OoD samples.”