Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness

By Zhenyi Wang, Tiehang Duan, Le Fang, Qiuling Suo, Mingchen Gao. 2021

21 citations recorded by the source corpus.

Recognizing new objects by learning from a few labeled examples in an evolving environment is crucial to obtain excellent generalization ability for real-world machine learning systems. A typical setting across current meta learning algorithms assumes a stationary task distribution during meta training. In this paper, we explore a more practical and challenging setting where task distribution changes over time with domain shift. Particularly, we consider realistic scenarios where task distribution is highly imbalanced with domain labels unavailable in nature. We propose a kernel-based method for domain change detection and a difficulty-aware memory management mechanism that jointly considers the imbalanced domain size and domain importance to learn across domains continuously. Furthermore, we introduce an efficient adaptive task sampling method during meta training, which significantly reduces task gradient variance with theoretical guarantees. Finally, we propose a challenging benchmark with imbalanced domain sequences and varied domain difficulty. We have performed extensive evaluations on the proposed benchmark, demonstrating the effectiveness of our method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Li Shen
2. Mingchen Gao
3. Zhenyi Wang
4. Qiuling Suo
5. Le Fang
6. Tiehang Duan
7. Dong Yu
8. Dongling Zhan

In Computer Vision and Pattern Recognition

1. Li Shen
2. Mingchen Gao
3. Zhenyi Wang
4. Qiuling Suo
5. Le Fang
6. Tiehang Duan
7. Dong Yu
8. Xiaoyang Wang

Citations in the corpus

• Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions (2022, methodology): “Furthermore, [62,63] has proposed a meta learning method to solve large-scale task stream with sequential domain shift.”
• Improving Task-free Continual Learning by Distributionally Robust Memory Evolution (2022, methodology): “Most existing CL methods (Lopez-Paz & Ranzato, 2017; Nguyen et al., 2018; Kirkpatrick et al., 2017; Zenke et al., 2017; von Oswald et al., 2019; Wang et al., 2021; Saha et al., 2021; Pham et al., 2021; Wang et al., 2022) are only applicable to the task-aware setting, where there are clear task…”
• Meta-learning without data via Wasserstein distributionally-robust model fusion (2022, background): “Online meta-learning (OML) [Finn et al., 2019a, Denevi et al., 2019, Yao et al., 2020, Babu et al., 2021, Wang et al., 2021, 2022] extends meta-learning to the online setting, where tasks sequentially arrives.”
• Learning to Learn and Remember Super Long Multi-Domain Task Sequence (2022, methodology): “Continual meta-learning [1, 13, 54, 74] is to apply the meta-learning techniques for continual learning.”