Decomposition-Based Transfer Distance Metric Learning for Image Classification

By Yong Luo, Tongliang Liu, Dacheng Tao, Chao Xu. 2014

118 citations recorded by the source corpus.

Distance metric learning (DML) is a critical factor for image analysis and pattern recognition. To learn a robust distance metric for a target task, we need abundant side information (i.e., the similarity/dissimilarity pairwise constraints over the labeled data), which is usually unavailable in practice due to the high labeling cost. This paper considers the transfer learning setting by exploiting the large quantity of side information from certain related, but different source tasks to help with target metric learning (with only a little side information). The state-of-the-art metric learning algorithms usually fail in this setting because the data distributions of the source task and target task are often quite different. We address this problem by assuming that the target distance metric lies in the space spanned by the eigenvectors of the source metrics (or other randomly generated bases). The target metric is represented as a combination of the base metrics, which are computed using the decomposed components of the source metrics (or simply a set of random bases); we call the proposed method, decomposition-based transfer DML (DTDML). In particular, DTDML learns a sparse combination of the base metrics to construct the target metric by forcing the target metric to be close to an integration of the source metrics. The main advantage of the proposed method compared with existing transfer metric learning approaches is that we directly learn the base metric coefficients instead of the target metric. To this end, far fewer variables need to be learned. We therefore obtain more reliable solutions given the limited side information and the optimization tends to be faster. Experiments on the popular handwritten image (digit, letter) classification and challenge natural image annotation tasks demonstrate the effectiveness of the proposed method.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Yun Raymond Fu
3. Tongliang Liu
4. Chao Xu
5. Chang Xu
6. Yunhe Wang
7. Ming Shao
8. Hongfu Liu

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Yun Raymond Fu
3. Yunhe Wang
4. Chang Xu
5. Tongliang Liu
6. Chao Xu
7. Qiang Yang
8. Ming Shao

Citations in the corpus

• Understanding How Feature Structure Transfers in Transfer Learning (2017, background): “…(sometimes called transfer learning [Maurer et al., 2013] or lifelong learning [Pentina and Lampert, 2014]), and learning to learn [Baxter, 2000] (sometimes called multi-task learning [Ando and Zhang, 2005]) have achieved great successes in a variety of tasks [Si et al., 2010; Luo et al., 2014].”
• Structure-Preserved Multi-source Domain Adaptation (2016, background): “Keywords-Transfer Learning; Multi-Source Domain Adaptation; Constraint Clustering I. INTRODUCTION As a widely discussed branch in transfer learning [26], domain adaptation has attracted lots of research attention recently [9], [18], [20].”
• General Heterogeneous Transfer Distance Metric Learning via Knowledge Fragments Transfer (2017, background): “Transfer learning [Pan and Yang, 2010; Luo et al., 2014; Hu et al., 2015; Isele et al., 2016] is able to mitigate this problem in the target learning task or domain by leveraging information from other related source tasks or domains [Liu et al., 2017].”
• Adversarial Learning of Portable Student Networks (2018, background): “…recently, i.e., directly learning a portable network (student network) with fewer parameters to inherit valuable properties of the original network (teacher network) (Hinton, Vinyals, and Dean 2015; Romero et al. 2014), which has the similar purpose to transfer learning (Luo et al. 2014; 2017).”
• Lifelong Metric Learning (2017, background): “ONLINE metric/similarity learning has received remarkable success in a variety of applications [1]–[3], such as data mining [4], information retrieval [5], and computer vision [6], [7], mainly due to its high efficiency and scalability to large-scale dataset.”