Online Structured Meta-learning

By Huaxiu Yao, Yingbo Zhou, Mehrdad Mahdavi, Zhenhui Jessie Li, Richard Socher, Caiming Xiong. 2020

29 citations recorded by the source corpus.

Learning quickly is of great importance for machine intelligence deployed in online platforms. With the capability of transferring knowledge from learned tasks, meta-learning has shown its effectiveness in online scenarios by continuously updating the model with the learned prior. However, current online meta-learning algorithms are limited to learn a globally-shared meta-learner, which may lead to sub-optimal results when the tasks contain heterogeneous information that are distinct by nature and difficult to share. We overcome this limitation by proposing an online structured meta-learning (OSML) framework. Inspired by the knowledge organization of human and hierarchical feature representation, OSML explicitly disentangles the meta-learner as a meta-hierarchical graph with different knowledge blocks. When a new task is encountered, it constructs a meta-knowledge pathway by either utilizing the most relevant knowledge blocks or exploring new blocks. Through the meta-knowledge pathway, the model is able to quickly adapt to the new task. In addition, new knowledge is further incorporated into the selected blocks. Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework in the context of both homogeneous and heterogeneous tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

CoMPS: Continual Meta Policy Search (2021, background): “Recent work has made progress toward using supervised meta-learning in an online setting (Ren et al., 2018; Finn et al., 2019; Khodak et al., 2019; Jerfel et al., 2019; Grant et al., 2019; Zhuang et al., 2019; Antoniou et al., 2020; Wallingford et al., 2020; Yu et al., 2020a; Yao et al., 2020).”
Learngene: From Open-World to Your Learning Task (2021, background): “Although meta continual learning (Yao et al. 2020; Gidaris and Komodakis 2018; Jerfel et al. 2019; Javed and White 2019; Gupta, Yadav, and Paull 2020) can solve the sequential task problem by recording the efﬁciency task buffer, it still cannot prevent data-interaction.”
Variable-Shot Adaptation for Online Meta-Learning (2020, methodology): “…(Finn et al., 2019; Grant et al., 2019; Khodak et al., 2019; Zhuang et al., 2019; Ren et al., 2020; Antoniou et al., 2020; Wallingford et al., 2020; Yao et al., 2020), we develop an algorithm that is specifically tailored to leverage variable amounts of data, allowing it to outperform a prior…”
Meta-learning without data via Wasserstein distributionally-robust model fusion (2022, background): “Online meta-learning (OML) [Finn et al., 2019a, Denevi et al., 2019, Yao et al., 2020, Babu et al., 2021, Wang et al., 2021, 2022] extends meta-learning to the online setting, where tasks sequentially arrives.”
Experts Collaboration Learning for Continual Multi-Modal Reasoning (2023, methodology): “Since only the contribution weights (ω) are updated during the above backward propagation, under the supervision of the loss function (L), these contribution weights will be optimized so that the most relevant (suitable) expert for solving the current task will obtain larger contribution weight [60]”