Transfer across Completely Different Feature Spaces via Spectral Embedding

By Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu. 2013

63 citations recorded by the source corpus.

In many applications, it is very expensive or time consuming to obtain a lot of labeled examples. One practically important problem is: can the labeled data from other related sources help predict the target task, even if they have 1) different feature spaces (e.g., image versus text data), 2) different data distributions, and 3) different output spaces? This paper proposes a solution and discusses the conditions where this is highly likely to produce better results. It first unifies the feature spaces of the target and source data sets by spectral embedding, even when they are with completely different feature spaces. The principle is to devise an optimization objective that preserves the original structure of the data, while at the same time, maximizes the similarity between the two. A linear projection model, as well as a nonlinear approach are derived on the basis of this principle with closed forms. Second, a judicious sample selection strategy is applied to select only those related source examples. At last, a Bayesian-based approach is applied to model the relationship between different output spaces. The three steps can bridge related heterogeneous sources in order to learn the target task. Among the 20 experiment data sets, for example, the images with wavelet-transformed-based features are used to predict another set of images whose features are constructed from color-histogram space; documents are used to help image classification, etc. By using these extracted examples from heterogeneous sources, the models can reduce the error rate by as much as 50 percent, compared with the methods using only the examples from the target task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Transfer Learning with Graph Co-Regularization (2012, methodology): “Recently, several methods have been proposed to explore the geometric structure for transfer learning [13], [14], [15], [16], [17].”
A Unified Framework for Metric Transfer Learning (2017, background): “source domain and the target domain [4], [14], [15], [16], [17] in order to reduce the distribution divergence between the source domain and the target domain.”
Instilling Social to Physical: Co-Regularized Heterogeneous Transfer Learning (2016, methodology): “To bridge the gap between different domains, the models including TLRisk (Dai et al. 2008), HTLIC (Zhu et al. 2011) that follows Collective Matrix Factorization (Singh and Gordon 2008), and HeMap (Shi et al. 2013), make use of direct correspondence data, e.g., tagged images that bridge images and te”
Adaptation Regularization: A General Framework for Transfer Learning (2014, background): “…factors between domains by preserving the important properties of the original data, e.g., statistical property [2], [25], geometric structure [26]–[28], or both [3]; 3) Distribution Adaptation, which learns a shared subspace where the distribution difference is explicitly reduced by minimizing…”
Transfer Knowledge between Cities (2016, methodology): “This work and its follow-up [16], however, can only handle the case where both source and target domains contain single modality only.”
Deep Model Based Domain Adaptation for Fault Diagnosis (2017, background): “[24]; 3) learning a subspace shared by the source and target domains, where the distributions discrepancy is significantly reduced via minimizing predefined distance measures, e.”
Transferable Contextual Bandit for Cross-Domain Recommendation (2018, background): “2011), HeMap (Shi et al. 2013), and CoHTL (Wei et al.”