Adaptive Deep Kernel Learning

By Prudencio Tossou, Basile Dura, François Laviolette, Mario Marchand, Alexandre Lacoste. 2019

31 citations recorded by the source corpus.

Deep kernel learning provides an elegant and principled framework for combining the structural properties of deep learning algorithms with the flexibility of kernel methods. By means of a deep neural network, it consists of learning a kernel operator which is combined with a differentiable kernel algorithm for inference. While previous work within this framework has mostly explored learning a single kernel for large datasets, we focus herein on learning a kernel family for a variety of tasks in few-shot regression settings. Compared to single deep kernel learning, our novel algorithm permits finding the appropriate kernel for each task during inference, rather than using the same for all tasks. As such, our algorithm performs more effectively with complex task distributions in few-shot learning, which we demonstrate by benchmarking against existing state-of-the-art algorithms using real-world, few-shot regression tasks related to drug discovery.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Kernel Continual Learning (2021, background): “Recently, they have also been demonstrated to be strong learners in tandem with deep neural networks (Wilson et al., 2016a;b; Tossou et al., 2019), especially when learning from limited data (Zhen et al., 2020; Patacchiola et al., 2020).”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “been adapted to the meta-learning setting [246], and is often coupled with Gaussian Processes [247].”
Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels (2020, methodology): “Tossou et al. (2019) have presented a variant of kernel learning for Gaussian Processes called Adaptive Deep Kernel Learning (ADKL), which ﬁnds a kernel for each task using a task encoder network.”
Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes (2020, methodology): “[32] consider Gaussian processes in the context of few-shot learning.”