Heterogeneous Transfer Learning for Image Classification

By Yingke Zhu, Yuqiang Chen, Zhongqi Lu, Sinno Jialin Pan, Gui-Rong Xue, Yong Yu, Qiang Yang. 2011

385 citations recorded by the source corpus.

Transfer learning as a new machine learning paradigm has gained increasing attention lately. In situations where the training data in a target domain are not sufficient to learn predictive models effectively, transfer learning leverages auxiliary source data from other related source domains for learning. While most of the existing works in this area only focused on using the source data with the same structure as the target data, in this paper, we push this boundary further by proposing a heterogeneous transfer learning framework for knowledge transfer between text and images. We observe that for a target-domain classification problem, some annotated images can be found on many social Web sites, which can serve as a bridge to transfer knowledge from the abundant text documents available over the Web. A key question is how to effectively transfer the knowledge in the source data even though the text can be arbitrarily found. Our solution is to enrich the representation of the target images with semantic concepts extracted from the auxiliary source data through a novel matrix factorization method. By using the latent semantic features generated by the auxiliary data, we are able to build a better integrated image classifier. We empirically demonstrate the effectiveness of our algorithm on the Caltech-256 image dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learned in Translation: Contextualized Word Vectors (2017, background): “Zhu et al. [2011] use matrix factorization to incorporate textual information into tagged images to enhance image classiﬁcation.”
A Comprehensive Survey on Transfer Learning (2019, methodology): “[149] utilizes a set of source images with tags and a set of documents to help construct the target classifier for images.”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, methodology): “3 that HDA methods often rely on an auxiliary intermediate multi-view domain [104, 106, 103, 101, 107, 109].”
Learning with Augmented Features for Heterogeneous Domain Adaptation (2012, methodology): “The same assumption was also used in (Yang et al., 2009; Zhu et al., 2011) for text-aid image clustering and classification.”
Heterogeneous Metric Learning with Joint Graph Regularization for Cross-Media Retrieval (2013, background): “Recently, learning from heterogeneous data has shown effectiveness in heterogeneous transfer learning(Zhu et al. 2011), heterogeneous multi task learning(Zhang and Yeung 2011), and transductive classification on heterogeneous information networks(Ji et al.”
Instilling Social to Physical: Co-Regularized Heterogeneous Transfer Learning (2016, methodology): “While this work tackled the clustering task, later a series of studies (Dai et al. 2008; Duan et al. 2012; Li et al. 2014; Shi et al. 2013; Wang and Mahadevan 2011; Zhu et al. 2011) focused on classification.”
A Decade Survey of Transfer Learning (2010–2020) (2020, methodology): “Unlike the previous methods [66], CT-Learn ﬁrst uses the co-occurrence information between the text data and image data to create a joint transition probability matrix P : (1 , 2) . . . λ 1 ,N P (1 ,N ) λ 2 , 1 P (1 , 1) λ 2 , 2 P (2 , 2) . . . λ 2 ,N P (2 ,N ) .”
Deep Asymmetric Transfer Network for Unbalanced Domain Adaptation (2018, methodology): “Most of existing methods (Zhu et al. 2011; Qi, Aggarwal, and Huang 2011; Shi et al. 2009; Dai et al. 2009) adopt shallow models attempting to explicitly reduce the discrepancy.”
Adaptation Regularization: A General Framework for Transfer Learning (2014, background): “As discussed in Section 2, our work is substantially different from a variety of prior cross-domain learning methods such as [4]–[6], [25], [37], which do not explicitly consider distribution matching or manifold regularization.”
Ieee Transactions on Knowledge and Data Engineering, Manuscropt Id 1 Social Recommendation with Cross-domain Transferable Knowledge (positive): “Transfer learning provides the key idea of using knowledge from auxiliary domains [30][31][32][33][34][35] and has been used in various ways.”
Transfer across Completely Different Feature Spaces via Spectral Embedding (2013, background): “The major idea is to take advantage of the image tags to build up the relationship between image and text, and further use text features to enrich image features to improve the learning accuracy.”
Visual and Semantic Knowledge Transfer for Large Scale Semi-Supervised Object Detection (2016, background): “general categories of TL have been proposed in previous work: homogeneous TL [13], [39], [40] in a single domain but with different data distributions in training and testing sets, and heterogeneous TL [41], [42], [43] across different domains or modalities.”
Heterogeneous Domain Adaptation for Multiple Classes (2014, methodology): “have been successfully applied to various real-world applications, such as WiFi-based localization [23, 24], image classification [35], video concept detection [14], sentiment analysis [5, 28], coreference resolution [32], vehicle routing [18], game playing [4], etc.”