Discriminative k-shot learning using probabilistic models

By Matthias Bauer, Mateo Rojas-Carulla, Jakub Bartlomiej Swiatkowski, Bernhard Schölkopf, Richard E. Turner. 2017

71 citations recorded by the source corpus.

This paper introduces a probabilistic framework for k-shot image classification. The goal is to generalise from an initial large-scale classification task to a separate task comprising new classes and small numbers of examples. The new approach not only leverages the feature-based representation learned by a neural network from the initial task (representational transfer), but also information about the classes (concept transfer). The concept information is encapsulated in a probabilistic model for the final layer weights of the neural network which acts as a prior for probabilistic k-shot learning. We show that even a simple probabilistic model achieves state-of-the-art on a standard k-shot learning dataset by a large margin. Moreover, it is able to accurately model uncertainty, leading to well calibrated classifiers, and is easily extensible and flexible, unlike many recent approaches to k-shot learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Meta & Continual Learning

Citations in the corpus

Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes (2019, methodology): “k-shot [26] Figure 4: Model design space.”
Meta-Learning Probabilistic Inference for Prediction (2018, methodology): “Approaches that employ pre-training and/or residual networks (Bauer et al., 2017; Qiao et al., 2017; Rusu et al., 2018; Gidaris and Komodakis, 2018; Oreshkin et al., 2018; Garcia and Bruna, 2017; Lacoste et al., 2018) have been excluded so that the quality of the learning algorithm can be assessed s”
Meta-Learning with Latent Embedding Optimization (2018, methodology): “Probabilistic meta-learning approaches such as those of Bauer et al. (2017) and Grant et al. (2018) have shown the advantages of learning Gaussian posteriors over model parameters.”
Uncertainty in Multitask Transfer Learning (2018, background): “Bauer et al. [3] use a neural network pre-trained on a standard multi-class dataset to obtain a good representation and use classes statistics to transfer prior knowledge to new classes.”
Bayesian Model-Agnostic Meta-Learning (2018, methodology): “In order to reduce the space and time complexity of the ensemble models (i.e., BMAML and EMAML) in this large network setting, we used the following parameter sharing scheme among the particles, similarly to Bauer et al. (2017).”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, methodology): “Differently, the feature extractor proposed in [1] is based on a ResNet-34 architecture with a reduced number of features.”
Meta-Transfer Learning Through Hard Tasks (2019, background): “67 Isotropic Gaussian, [51] ResNet-34 (pre) 56.”
Variational Few-Shot Learning (2019, background): “The best performance is indicated in bold . as widely adopted in [32, 34, 2].”
Teachable machines for accessibility (2017, background): “By leveraging prior work in transfer learning [16, 17] and k-shot learning [1, 25], we are now able to build robust systems that learn to represent a task in an abstract way and transfer this knowledge to new, previously unseen tasks given only one or few examples of the novel task.”
SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning (2019, methodology): “We compare our baselines to a range of state-of-the-art few-shot learners [1, 4, 5, 6, 8, 10, 15, 17, 21, 22, 23, 24, 26, 29, 25, 30, 31, 32, 34, 36].”
Relational Generalized Few-Shot Learning (2019, methodology): “We adopt the cosine classifier since it was found to be preferable over the originally proposed L2 distance [6] when combining existing and novel classes [15, 10].”
Self-Supervised Learning for Few-Shot Image Classification (2019, methodology): “Current methods [9, 10, 8, 11] with good performance mostly apply a ResNet12 [12] or a wide ResNet [13] as the embedding network and surpass the methods [14, 15] with deeper network.”
Adaptive Cross-Modal Few-Shot Learning (2019, unknown): “5-way 5-shot 5-way 10-shot Uni-modality few-shot learning baselines Matching Network [53] 43.56 0.84% 55.31 0.73% - Prototypical Network [47] 49.42 0.78% 68.20 0.66% 74.30 0.52% Discriminative k-shot [2] 56.30 0.40% 73.90 0.30% 78.50 0.00% Meta-Learner LSTM [38] 43.44 0.77% 60.60 0.71% - MAML [7] 48”
Hyperbolic Image Embeddings (2019, background): “Most of the existing state-ofthe-art few–shot learning models are based on metric learning approaches, utilizing the distance between image representations computed by deep neural networks as a measure of similarity [53, 43, 48, 28, 4, 6, 23, 2, 38, 5].”