Infrared-Visible Cross-Modal Person Re-Identification with an X Modality

By Diangang Li, Xing Wei, Xiaopeng Hong, Yihong Gong. 2020

430 citations recorded by the source corpus.

This paper focuses on the emerging Infrared-Visible cross-modal person re-identification task (IV-ReID), which takes infrared images as input and matches with visible color images. IV-ReID is important yet challenging, as there is a significant gap between the visible and infrared images. To reduce this ‘gap’, we introduce an auxiliary X modality as an assistant and reformulate infrared-visible dual-mode cross-modal learning as an X-Infrared-Visible three-mode learning problem. The X modality restates from RGB channels to a format with which cross-modal learning can be easily performed. With this idea, we propose an X-Infrared-Visible (XIV) ReID cross-modal learning framework. Firstly, the X modality is generated by a lightweight network, which is learnt in a self-supervised manner with the labels inherited from visible images. Secondly, under the XIV framework, cross-modal learning is guided by a carefully designed modality gap constraint, with information exchanged cross the visible, X, and infrared modalities. Extensive experiments are performed on two challenging datasets SYSU-MM01 and RegDB to evaluate the proposed XIV-ReID approach. Experimental results show that our method considerably achieves an absolute gain of over 7% in terms of rank 1 and mAP even compared with the latest state-of-the-art methods.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Class-Incremental Learning (2020, background): “Convolutional Neural Networks (CNNs) have been successfully applied to a broad range of computer vision tasks [17, 11, 34, 4, 22, 25, 45, 19].”
Channel Augmented Joint Learning for Visible-Infrared Recognition (2021, background): “However, the image generation process usually needs additional computational cost and suffers from unavoidable noise [21].”
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification (2020, methodology): “An intermediate X-modality is designed in [19] to address the modality discrepancy.”