Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learning
By Kun Yan, Zied Bouraoui, Ping Wang, Shoaib Jameel, Steven Schockaert. 2021
36 citations recorded by the source corpus.
Few-shot learning (FSL) is the task of learning to recognize previously unseen categories of images from a small number of training examples. This is a challenging task, as the available examples may not be enough to unambiguously determine which visual features are most characteristic of the considered categories. To alleviate this issue, we propose a method that additionally takes into account the names of the image classes. While the use of class names has already been explored in previous work, our approach differs in two key aspects. First, while previous work has aimed to directly predict visual prototypes from word embeddings, we found that better results can be obtained by treating visual and text-based prototypes separately. Second, we propose a simple strategy for learning class name embeddings using the BERT language model, which we found to substantially outperform the GloVe vectors that were used in previous work. We furthermore propose a strategy for dealing with the high dimensionality of these vectors, inspired by models for aligning cross-lingual word embeddings. We provide experiments on miniImageNet, CUB and tieredImageNet, showing that our approach consistently improves the state-of-the-art in metric-based FSL.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ekin Akyürek
- Afra Feyza Akyürek
- Jacob Andreas
- Derry Tanti Wijaya
- Mohit Bansal
- Trevor Darrell
- Eric Eaton
- Marco Baroni
In Computer Vision and Pattern Recognition
- Afra Feyza Akyürek
- Ekin Akyürek
- Jacob Andreas
- Derry Tanti Wijaya
- Trevor Darrell
- Mohit Bansal
- Luc Van Gool
- Dacheng Tao
Citations in the corpus
- Subspace Regularizers for Few-Shot Class Incremental Learning (2021, background): “…(especially language) to bootstrap learning of new classes is widely studied (Frome et al., 2013; Radford et al., 2021; Reed et al., 2016; Yan et al., 2021)—particularly in the zero-shot learning and generalized zero-shot learning where side information is the only source of information…”