Multi-Task CNN Model for Attribute Prediction

By Abrar H. Abdulnabi, G. Wang, Jiwen Lu, K. Jia. 2015

310 citations recorded by the source corpus.

This paper proposes a joint multi-task learning algorithm to better predict attributes in images using deep convolutional neural networks (CNN). We consider learning binary semantic attributes through a multi-task CNN model, where each CNN will predict one binary attribute. The multi-task learning allows CNN models to simultaneously share visual knowledge among different attribute categories. Each CNN will generate attribute-specific feature representations, and then we apply multi-task learning on the features to predict their attributes. In our multi-task framework, we propose a method to decompose the overall model's parameters into a latent task matrix and combination matrix. Furthermore, under-sampled classifiers can leverage shared statistics from other classifiers to improve their performance. Natural grouping of attributes is applied such that attributes in the same group are encouraged to share more knowledge. Meanwhile, attributes in different groups will generally compete with each other, and consequently share less knowledge. We show the effectiveness of our method on two popular attribute datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

An Overview of Multi-task Learning (2018, background): “…images include object categorization [86], image segmentation [89,90], identifying brain imaging predictors [91], saliency detection [92], action recognition [93], scene classification [94], multi-attribute prediction [95], multi-camera person re-identification [96], and immediacy prediction [97].”
Cross-Stitch Networks for Multi-task Learning (2016, methodology): “So as our final baseline, we compare to a variant of [1, 62] by adapting their method to our setting and report this as ‘MTL-shared’.”
A Survey on Multi-Task Learning (2017, methodology): “In [176], a multi-task convolutional neural network (CNN), which has individual CNNs for each task and fuses different CNNs in a common layer via a sparse transformation, is proposed for imagebased multi-label attribute prediction.”
MarioQA: Answering Questions by Watching Gameplay Videos (2016, background): “…networks trained on large-scale datasets have been making signiﬁcant progress on various visual recognition problems, most of these tasks focus on the recognition in the same or similar levels, e.g ., objects [16, 17], scenes [27], actions [15, 19], attributes [2, 26], face identities [21], etc.”
Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition (2017, methodology): “multaneously to boost the performance of the main task or all tasks. It has been successfully applied to face detection [6], [55], face alignment [58], pedestrian detection [47], attribute estimation [1], and so on. Despite the success of MTL in various vision problems, there is a lack of comprehens”
Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach (2017, methodology): “• Unlike existing methods that have focused on face alignment, human pose estimation, and human attribute estimation [20], [47], [48], the proposed approach focuses on joint estimation of multiple attributes from a face image.”