Learning to Learn Kernels with Variational Random Features

By Xiantong Zhen, Hao Sun, Yingjun Du, Jun Xu, Yilong Yin, Ling Shao, Cees G. M. Snoek. 2020

38 citations recorded by the source corpus.

In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational random features (MetaVRF) to learn adaptive kernels for the base-learner, which is developed in a latent variable model by treating the random feature basis as the latent variable. We formulate the optimization of MetaVRF as a variational inference problem by deriving an evidence lower bound under the meta-learning framework. To incorporate shared knowledge from related tasks, we propose a context inference of the posterior, which is established by an LSTM architecture. The LSTM-based inference network can effectively integrate the context information of previous tasks with task-specific information, generating informative and adaptive features. The learned MetaVRF can produce kernels of high representational power with a relatively low spectral sampling rate and also enables fast adaptation to new tasks. Experimental results on a variety of few-shot regression and classification tasks demonstrate that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.

Subfields: Artificial Intelligence, Cancer Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Xudong Zhang
4. Yandan Yang
5. Su Lu
6. Haochen Wang
7. Xiantong Zhen
8. Xianbin Cao

In Cancer Research

1. De-chuan Zhan
2. Han-Jia Ye

Citations in the corpus

• Few-Shot Semantic Segmentation with Democratic Attention Networks (2020, background): “Few-shot learning has been investigated under the meta-learning framework by exploring shared metric/similarity space [31, 35, 33, 42], or optimization algorithms [26, 6, 23].”
• Tailoring Embedding Function to Heterogeneous Few-Shot Tasks by Global and Local Feature Adaptors (2021, background): “As in many few-shot learning works (Li et al. 2019; Ye et al. 2020; Zhen et al. 2020), we pre-train our embedding network on the meta-training split using cross-entropy loss function.”