Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

By Xi Yin, Xiaoming Liu. IEEE Transactions on Image Processing, 2017

306 citations recorded by the source corpus.

This paper explores multi-task learning (MTL) for face recognition. First, we propose a multi-task convolutional neural network (CNN) for face recognition, where identity classification is the main task and pose, illumination, and expression (PIE) estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss weights to each side task, which solves the crucial problem of balancing between different tasks in MTL. Third, we propose a pose-directed multi-task CNN by grouping different poses to learn pose-specific identity features, simultaneously across all poses in a joint framework. Last but not least, we propose an energy-based weight analysis method to explore how CNN-based MTL works. We observe that the side tasks serve as regularizations to disentangle the PIE variations from the learnt identity features. Extensive experiments on the entire multi-PIE dataset demonstrate the effectiveness of the proposed approach. To the best of our knowledge, this is the first work using all data in multi-PIE for face recognition. Our approach is also applicable to in-the-wild data sets for pose-invariant face recognition and achieves comparable or better performance than state of the art on LFW, CFP, and IJB-A datasets.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/79673a7e28237338792aff809d2ba04f92f9da1b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Data Uncertainty Learning in Face Recognition (2020, unknown): “] 0.7M 99:28 94:9 65:23 76:52 SphereFace [27] 0.5M 99:42 95:0 75:77 89:14 ArcFace [7] 5.8M 99:83 98:02 81:03 96:98 CosFace [41] 5M 99 :73 97 6 77 11 89 88 L2-Face [32] 3.7M 99:78 96:08 - - Yin et al. [49] 1M 98:27 - - 94:39 PFE [35] 4.4M 99:82 97:36 78:95 93:34 Baseline 3.6M 99 :83 96 50 98 30 98 75”
Feature Transfer Learning for Deep Face Recognition with Under-Represented Data (2018, methodology): “feature transfer learning further improves the performance signicantly. On LFW, our performance is among the Feature Transfer Learning for Deep Face Recognition with Long-Tail Data 13 Test ! LFW MTL [59] 98:27 L-Softmax [42] 98:71 VGG Face [19] 98:95 DeepID2 [17] 99:15 NormFace [27] 99:19 CenterLoss”
Thin-Plate Spline Motion Model for Image Animation (2022, background): “Multi-scale feature fusion has proven to be effective in several tasks in computer vision, including keypoint prediction [4, 29, 33], segmentation [19, 24, 36] and image generation [8, 15, 16].”
Towards Large-Pose Face Frontalization in the Wild (2017, background): “Pose-invariant feature disentanglement [20] or identity preservation [45, 39] methods aim to factorize out the non-identity part with care-fully designed networks.”
Mitigating Bias in Gender, Age and Ethnicity Classification: A Multi-task Convolution Neural Network Approach (2018, methodology): “, [28], obtaining weights via brute-force search [28] or by dynamically assigning disjoint weights for the side task [29].”
Probabilistic Face Embeddings (2019, unknown): “tried implementing ArcFace [5] but it does not converge well in our case. So we did not use it. Method Training Data IJB-A (TAR@FAR) CFP-FP 0:1% 1:0% DR-GAN [36] 1M 53:9 4:3 77:4 2:7 93:41 Yin et al. [50] 0.5M 73:9 4:2 77:5 2:5 94:39 TPE [30] 0.5M 90:0 1:0 93:4 0:5 89:17 NAN [47] 3M 88:1 1:1 94:1 0:”
Deep Face Recognition: A Survey (2018, background): “According to their characteristics, we divide these scenes into four categories: cross-factor FR, heterogenous FR, multiple (or single) media FR and FR in industry (Fig.”