GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning

By Idan Achituve, Aviv Navon, Yochai Yemini, Gal Chechik, Ethan Fetaya. 2021

42 citations recorded by the source corpus.

Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the P\'olya Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.

Subfields: Artificial Intelligence, Environmental Engineering, Probabilistic & Bayesian Methods, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Jun Zhu
6. Liyuan Wang
7. Xingxing Zhang
8. Fu-Yun Wang

In Environmental Engineering

1. De-chuan Zhan

In Probabilistic & Bayesian Methods

1. Liyuan Wang
2. Jun Zhu
3. De-chuan Zhan
4. Hang Su
5. Han-Jia Ye
6. Xingxing Zhang

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Jun Zhu
8. Fu-Yun Wang

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “[490]), developing a tree-based hierarchical classifier with Gaussian processes (GP-Tree [6]), performing hyperdimen-”
• Forward Compatible Few-Shot Class-Incremental Learning (2022, background): “Few-Shot Class-Incremental Learning: is recently proposed to address the few-shot inputs in the incremental learning scenario [1, 11, 24, 61].”
• A Survey on Few-Shot Class-Incremental Learning (2023, methodology): “To address the limitations of common Gaussian process classification in large-scale class classifi-6 cation tasks, Achituve et al. (2021) proposed GP-Tree.”
• SemGIF: A Semantics Guided Incremental Few-shot Learning Framework with Generative Replay (2021, background): “It is also worth mentioning that a related but different problem setup from IFSL is few-shot class incremental learning (FSCIL) [1, 19, 36, 46] consisting of a large number of incremental episodes with each stage accumulating knowledge from a small number of classes under limited supervision.”