Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition

By Ran He, Xiang Wu, Zhenan Sun, Tien-Ping Tan. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017

297 citations recorded by the source corpus.

Heterogeneous face recognition (HFR) aims at matching facial images acquired from different sensing modalities with mission-critical applications in forensics, security and commercial sectors. However, HFR presents more challenging issues than traditional face recognition because of the large intra-class variation among heterogeneous face images and the limited availability of training samples of cross-modality face image pairs. This paper proposes the novel Wasserstein convolutional neural network (WCNN) approach for learning invariant features between near-infrared (NIR) and visual (VIS) face images (i.e., NIR-VIS face recognition). The low-level layers of the WCNN are trained with widely available face images in the VIS spectrum, and the high-level layer is divided into three parts: the NIR layer, the VIS layer and the NIR-VIS shared layer. The first two layers aim at learning modality-specific features, and the NIR-VIS shared layer is designed to learn a modality-invariant feature subspace. The Wasserstein distance is introduced into the NIR-VIS shared layer to measure the dissimilarity between heterogeneous feature distributions. W-CNN learning is performed to minimize the Wasserstein distance between the NIR distribution and the VIS distribution for invariant deep feature representations of heterogeneous face images. To avoid the over-fitting problem on small-scale heterogeneous face data, a correlation prior is introduced on the fully-connected WCNN layers to reduce the size of the parameter space. This prior is implemented by a low-rank constraint in an end-to-end network. The joint formulation leads to an alternating minimization for deep feature representation at the training stage and an efficient computation for heterogeneous data at the testing stage. Extensive experiments using three challenging NIR-VIS face recognition databases demonstrate the superiority of the WCNN method over state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/04fbf11a121beef28338e18815a878deefa338a9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Weihong Deng
2. Mei Wang
3. Ran He
4. Xiaogang Wang
5. Zhen Lei
6. S. Li
7. Zhenan Sun
8. Xiangyu Zhu

In Signal Processing

1. Ran He
2. Mei Wang
3. Weihong Deng
4. Xiang Wu
5. Zhenan Sun
6. Zhen Lei
7. S. Li
8. Chenxu Zhao

Citations in the corpus

• Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer (2020, background): “Cross-modality retrieval has been widely studied for heterogeneous face recognition [15] and textto-image retrieval [9, 16, 21, 22, 27].”
• Learning an Evolutionary Embedding via Massive Knowledge Distillation (2020, methodology): “Following the setting in [15], 20 identities are used for testing.”
• A Survey of Deep Learning-Based Object Detection (2019, background): “[128] propose a novel Wasserstein convolutional neural network approach to learn invariant features between near-infrared (NIR) and visual (VIS) face images.”
• Deep Face Recognition: A Survey (2018, background): “[238], [239] divided the high layer of the network into a NIR layer, a VIS layer and a NIR-VIS shared layer, then, a modality-invariant feature can be learned by the NIR-VIS shared layer.”
• Learning Meta Face Recognition in Unseen Domains (2020, methodology): “(ii) even with such a huge modality gap, our performance is comparable to several CNN-based methods [43, 44], which use MS-Celeb for pre-training and the target domain NIR-VIS dataset for fine-tuning.”