Learning using privileged information: SV M+ and weighted SVM

By Maksim Lapin, Matthias Hein, Bernt Schiele. 2013

142 citations recorded by the source corpus.

Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training. The same goal is pursued within the learning using privileged information paradigm which was recently introduced by Vapnik et al. and is aimed at utilizing additional information available only at training time-a framework implemented by SVM+. We relate the privileged information to importance weighting and show that the prior knowledge expressible with privileged features can also be encoded by weights associated with every training example. We show that a weighted SVM can always replicate an SVM+ solution, while the converse is not true and we construct a counterexample highlighting the limitations of SVM+. Finally, we touch on the problem of choosing weights for weighted SVMs when privileged features are not available.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Adversarial Domain Adaptation (2017, methodology): “In addition to these methods, [6, 36, 50] have addressed UDA when an auxiliary data view [31, 37], is available during training, but that is beyond the scope of this work.”
Unifying distillation and privileged information (2015, background): “z et al., 2014), clustering (Feyereisl & Aickelin, 2012), metric learning (Fouad et al., 2013), Gaussian process classiﬁcation (Hern´andez-Lobato et al., 2014), and ﬁnance (Ribeiro et al., 2010). Lapin et al. (2014) show that learning using privileged information is a particular instance of impo”
Unified Deep Supervised Domain Adaptation and Generalization (2017, background): “Recently, [7, 42] have addressed UDA when an auxiliary data view [36, 43], is available during training, which is beyond the scope of this work.”
Sidekick Policy Learning for Active Visual Exploration (2018, background): “In learning with privileged information, an “expert” provides the student with training data having extra information (unavailable during testing) [61,53,37].”
Simple and Efficient Learning using Privileged Information (2016, methodology): “The SVM+ at-tracts much attention recently [Vapnik and Izmailov, 2015; Lopez-Paz et al. , 2015; Lapin et al. , 2014], and has been successfully applied to different applications [Niu et al. , 2016; Sharmanska et al. , 2013; Fouad et al. , 2012].”
Transfer Hashing with Privileged Information (2016, background): “Inspired by the exciting results of LUPI, which prove that the amount of training data can be significantly reduced with privileged information for training a precise predictive model [Lapin et al., 2014; Vapnik and Vashist, 2009; Vapnik and Izmailov, 2015], we propose a new framework”
Learning With Single-Teacher Multi-Student (2018, methodology): “This formulation is similar with the idea of weighted SVM (Lapin, Hein, and Schiele 2014), where each of example is assigned an importance weight d−1i .”
Information Bottleneck Learning Using Privileged Information for Visual Recognition (2016, methodology): “Figure 3 shows the differences between the accuracies of LMIBPI versus RankTr, SVM+, SVM2k-LUPI and KCCA-LUPI.”
Graph Distillation for Action Detection with Privileged Modalities (2017, background): “, however, the privileged information is not available, in which case the Student operates without supervision of the Teacher [55]. Several work employed privileged information (PI) on SVM classiﬁers [55,23,54,59,12]. Recently, privileged information has been applied to deep learning in various sett”
Classifier learning with hidden information (2015, background): “Besides, [13] showed that SVM+ can be formulated as a special case of instance weighted SVM.”
Adaptive SVM+: Learning with Privileged Information for Domain Adaptation (2017, background): “[16] related the privileged information framework to the importance of sample weighting and showed that prior knowledge can be encoded using weights in a regular SVM.”
Information Bottleneck Domain Adaptation with Privileged Information for Visual Recognition (2016, methodology): “One LUPI implementation is the SVM+ [45, 29], later extended to a learning to rank approach in [38], where it is shown that different types of auxiliary information, such as bounding boxes, attributes, and text can be used for learning a better classifier for object recognition.”
From Dependence to Causation (2016, background): “The framework of learning using privileged information enjoys theoretical analysis (Pechyony and Vapnik, 2010), equivalence analysis to weighted learning (Lapin et al., 2014), and multiple applications that include ranking (Sharmanska et al.”
Towards Knowledgeable Supervised Lifelong Learning Systems (2020, methodology): “Lapin et al. (2014) analysed the equivalence of the solutions achieved by solving the problems of training an SVM hypothesis with weighted training examples and SVM+ (Pechyony et al., 2010) as the SVM implementation of the idea of learning using privileged information (Vapnik & Vashist, 2009).”