Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices

By Jie Wang, Jieping Ye. 2015

18 citations recorded by the source corpus.

Multi-task feature learning (MTFL) is a powerful technique in boosting the predictive performance by learning multiple related classification/ regression/clustering tasks simultaneously. However, solving the MTFL problem remains challenging when the feature dimension is extremely large. In this paper, we propose a novel screening rule--that is based on the dual projection onto convex sets (DPC)--to quickly identify the inactive features--that have zero coefficients in the solution vectors across all tasks. One of the appealing features of DPC is that: it is safe in the sense that the detected inactive features are guaranteed to have zero coefficients in the solution vectors across all tasks. Thus, by removing the inactive features from the training phase, we may have substantial savings in the computational cost and memory usage without sacrificing accuracy. To the best of our knowledge, it is the first screening rule that is applicable to sparse models with multiple data matrices. A key challenge in deriving DPC is to solve a nonconvex problem. We show that we can solve for the global optimum efficiently via a properly chosen parametrization of the constraint set. Moreover, DPC has very low computational cost and can be integrated with any existing solvers. We have evaluated the proposed DPC rule on both synthetic and real data sets. The experiments indicate that DPC is very effective in identifying the inactive features--especially for high dimensional data--which leads to a speedup up to several orders of magnitude.

Subfields: Artificial Intelligence, Computational Mechanics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Yu Zhang
3. Tao Li
4. Huan Liu
5. Yun Li
6. Mingsheng Long
7. Jianmin Wang
8. Philip S. Yu

In Computational Mechanics

1. Qiang Yang
2. Yu Zhang
3. Tao Li
4. Huan Liu
5. Yun Li

Citations in the corpus

• A Survey on Multi-Task Learning (2017, methodology): “Moreover, in order to make speedup, a safe screening method is proposed in [30] to filter out useless features corresponding to zero rows in W before optimizing problem (4).”
• Recent advances in feature selection and its applications (2017, methodology): “The safe screening rule has been applied into multi-task feature learning [46].”