Learning the Shared Subspace for Multi-task Clustering and Transductive Transfer Classification

By Quanquan Gu, Jie Zhou. 2009

167 citations recorded by the source corpus.

There are many clustering tasks which are closely related in the real world, e.g. clustering the web pages of different universities. However, existing clustering approaches neglect the underlying relation and treat these clustering tasks either individually or simply together. In this paper, we will study a novel clustering paradigm, namely multi-task clustering, which performs multiple related clustering tasks together and utilizes the relation of these tasks to enhance the clustering performance. We aim to learn a subspace shared by all the tasks, through which the knowledge of the tasks can be transferred to each other. The objective of our approach consists of two parts: (1) Within-task clustering: clustering the data of each task in its input space individually; and (2) Cross-task clustering: simultaneous learning the shared subspace and clustering the data of all the tasks together. We will show that it can be solved by alternating minimization, and its convergence is theoretically guaranteed. Furthermore, we will show that given the labels of one task, our multi-task clustering method can be extended to transductive transfer classification (a.k.a. cross-domain classification, domain adaption). Experiments on several cross-domain text data sets demonstrate that the proposed multi-task clustering outperforms traditional single-task clustering methods greatly. And the transductive transfer classification method is comparable to or even better than several existing transductive transfer classification approaches.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Domain Generalization via Invariant Feature Representation (2013, background): “For example, a typical approach in multitask learning is to uncover a joint (latent) feature/subspace that beneﬁts tasks individually (Ar-gyriou et al. 2007, Gu and Zhou 2009, Passos et al. 2012).”
An introduction to domain adaptation and transfer learning (2018, background): “The functional relationship between the input and the classes can be preserved by incorporating a central subspace in which the input and the classes are conditionally independent [89,118].”
Smart Multitask Bregman Clustering and Multitask Kernel Clustering (2015, background): “In transductive transfer learning, there are no labeled data in the target domain, while a lot of labeled data in the source domain [Arnold et al. 2007; Dai et al. 2007a; Gu and Zhou 2009; Ling et al. 2008; Pan et al. 2008].”
Multitask Bregman clustering (2010, background): “(Gu and Zhou 2009) handled the clustering problem by learning a shared subspace among tasks.”
Cross-Stitch Networks for Multi-task Learning (2016, background): “However, most of such work, with publicly available code, formulates multi-task learning in an optimization framework that requires all data points in memory [6, 14, 23, 34, 49, 60, 61].”
Self-Adapted Multi-Task Clustering (2016, methodology): “We also compare SAMTC with the multi-task clustering methods which work for completely related tasks: the shared subspace learning multi-task clustering (LSSMTC) method [Gu and Zhou, 2009] and the convex discriminative multi-task feature clustering (MTFC) method [Zhang, 2015].”
Deep Bilinear Learning for RGB-D Action Recognition (2018, background): “To mine more interactive information among multi-modal features, lots of methods are proposed to explicitly learn shared-specific structures among features [11, 13, 28].”
Multi-Task Multi-View Clustering (2016, unknown): “For another example, the tasks for clustering the web images collected from two web sites are two related tasks.”
Multi-Task Clustering with Model Relation Learning (2018, methodology): “We also compare MTCM-RL with the multi-task clustering methods: the shared sub-space learning multi-task clustering (LSSMTC) method [Gu and Zhou, 2009], the smart multi-task Bregman and Kernel clustering (S-MBC and S-MKC) methods [Zhang et al. , 2015], the convex discriminative multi-task feature…”
Multi-view clustering: A survey (2018, unknown): “Later on, Zhang and Zhou[165] relaxed these assumptions, and introduced an improved cross-domain multi-task clustering, which can performs multiple related clustering tasks simultaneously through domain adaptation.”
Multi-Task Multi-View Clustering for Non-Negative Data (2015, methodology): “More specifically, if there are some tasks related to each other, there may exist some common latent features that cause the distributions of the related tasks to be close to each other [Gu and Zhou, 2009].”
Smart Multi-Task Bregman Clustering and Multi-Task Kernel Clustering (2013, background): “(Gu and Zhou 2009) learned a subspace shared by multiple related tasks.”
Convex Discriminative Multitask Clustering (2013, background): “They are 1) Learning the Shared Subspace for MTC (LSSMTC) [39], 2) Learning a Spectral Kernel for MTC (LSKMTC) [42], and 3) Multitask Bregman Clustering with Pairwise task regularization (MBC-P) [41].”
Learning a Kernel for Multi-Task Clustering (2011, methodology): “Recently, multi-task clustering (Gu and Zhou 2009) was proposed, which clusters the data from related tasks simultaneously.”