Transfer learning for image classification with sparse prototype representations

By Ariadna Quattoni, Michael Collins, Trevor Darrell. 2008

279 citations recorded by the source corpus.

To learn a new visual category from few examples, prior knowledge from unlabeled data as well as previous related categories may be useful. We develop a new method for transfer learning which exploits available unlabeled data and an arbitrary kernel function; we form a representation based on kernel distances to a large set of unlabeled data points. To transfer knowledge from previous related problems we observe that a category might be learnable using only a small subset of reference prototypes. Related problems may share a significant number of relevant prototypes; we find such a concise representation by performing a joint loss minimization over the training sets of related problems with a shared regularization penalty that minimizes the total number of prototypes involved in the approximation. This optimization problem can be formulated as a linear program that can be solved efficiently. We conduct experiments on a news-topic prediction task where the goal is to predict whether an image belongs to a particular news topic. Our results show that when only few examples are available for training a target topic, leveraging knowledge learnt from other topics can significantly improve performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition (2013, background): “Our work is also related to representation learning schemes in computer vision which form an intermediate representation based on learning classifiers on related tasks (Li et al., 2010; Torresani et al., 2010; Quattoni et al., 2008).”
Cross-Stitch Networks for Multi-task Learning (2016, background): “and their widespread applications in other fields, such as genomics [38], natural language processing [7, 8, 35] and computer vision [3, 10, 30, 31, 40, 51, 53, 58].”
Recognizing indoor scenes (2009, methodology): “For this we will use the Gist descriptor using the code available online [10].”
Learning to share visual appearance for multiclass object detection (2011, background): “Other models learn the relations between categories [30, 21, 22, 1, 25, 13, 29].”
Learning Categories From Few Examples With Multi Model Knowledge Transfer (2014, background): “Several other works in the computer vision literature followed this ﬁrst attempt [8], [9], [10], [11], introducing different methods to increase the categorization performance with respect to learning from scratch in case of few available samples.”
Self-Supervised Pretraining Improves Self-Supervised Pretraining (2021, background): “Transfer learning studies how a larger, more general, or more specialized source dataset can be leveraged to improve performance on target downstream datasets/tasks [47, 44, 2, 10, 22, 20, 13, 15, 65, 18, 31, 45].”
Heterogeneous Transfer Learning for Image Classification (2011, background): “Recently, transfer learning methods have been proposed to use knowledge from auxiliary data in a different but re-lated domain to help learn the target tasks (Wu and Diet-terich 2004; Mihalkova et al. 2007; Quattoni et al. 2008; Daum´e 2007).”
Beyond spatial pyramids: Receptive field learning for pooled image features (2012, methodology): “To this end, we adopt the idea of structured sparsity [26, 29], and train a multiple-class linear classifier y = f(x) = Wx+b via the following optimization problem:”
Multiclass transfer learning from unconstrained priors (2011, background): “Many works address the issue of what to transfer (samples [3], feature representation [23], model parameters [13, 29, 31]), some focus on how to transfer (generative approaches [13, 29], boosting [36], KNN [27] and Support Vector Machine (SVM) [10, 31]), while others concentrate on how to avoid nega”
Cross-view action recognition via view knowledge transfer (2011, background): “Moreover, some works [31, 9, 10, 11] have used concepts from transfer learning for addressing object and image classification problems.”
Learning with Whom to Share in Multi-task Feature Learning (2011, background): “Copyright 2011 by the author(s)/owner(s). ing those in computer vision (Torralba et al., 2007; Loeff & Farhadi, 2008; Quattoni et al., 2008), natural language processing (Ando & Zhang, 2005), and genomics (Obozinski et al., 2009).”
Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships (2009, methodology): “Such Prototype models have been successfully used for object recognition [15, 16].”
Heterogeneous Face Recognition Using Kernel Prototype Similarities (2013, methodology): “Later, Liu et al. performed the transformation using local linear embedding to estimate the corresponding photo patch from a sketch patch [4].”
Model recommendation: Generating object detectors from few samples (2015, background): “, sparse prototype representations [43], hypothesis transfer learning [27, 49], regularized SVM [4]), multi-task learning (e.”
LabelMe: Online Image Annotation and Applications (2010, background): “The presence of the heavy tailed distribution of obje ct counts in the LabelMe dataset is important to encourage the development of algorithms that can learn from few training samples by transferring knowledge from other, more frequen t, object categories [14], [73], [4], [65], [45], [44].”
Weakly Supervised Localization and Learning with Generic Knowledge (2012, unknown): “2007; Thrun 1996) in computer vision, where learning the new class (target) is helped by labeled examples of other related classes (sources) (Deselaers et al. 2010; Fei-Fei et al. 2004; Lampert et al. 2009a; Lando and Edelman 1995; Quattoni et al. 2008; Rohrbach et al. 2010; Stark et al. 2009; Tomma”
KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection (2017, background): “The number of images is sufficient to be used for different tasks, e.g., image retrieval, machine learning, deep learning and transfer learning, etc. [2, 12, 28].”
Similarity Scores Based on Background Samples (2009, background): “It differs from flavors of transfer learning that use unlabeled samples [16], since they use separate supervised learning tasks in order to benefit from the unlabeled set.”