Cross-Modality Person Re-Identification with Generative Adversarial Training

By Pingyang Dai, Rongrong Ji, Haibin Wang, Qiong Wu, Yuyu Huang. International Joint Conference on Artificial Intelligence, 2018

459 citations recorded by the source corpus.

Person re-identification (Re-ID) is an important task in video surveillance which automatically searches and identifies people across different cameras. Despite the extensive Re-ID progress in RGB cameras, few works have studied the Re-ID between infrared and RGB images, which is essentially a cross-modality problem and widely encountered in real-world scenarios. The key challenge lies in two folds, i.e., the lack of discriminative information to re-identify the same person between RGB and infrared modalities, and the difficulty to learn a robust metric towards such a large-scale cross-modality retrieval. In this paper, we tackle the above two challenges by proposing a novel cross-modality generative adversarial network (termed cmGAN). To handle the issue of insufficient discriminative information, we leverage the cutting-edge generative adversarial training to design our own discriminator to learn discriminative feature representation from different modalities. To handle the issue of large-scale cross-modality metric learning, we integrates both identification loss and cross-modality triplet loss, which minimize inter-class ambiguity while maximizing cross-modality similarity among instances. The entire cmGAN can be trained in an end-to-end manner by using standard deep neural network framework. We have quantized the performance of our work in the newly-released SYSU RGB-IR Re-ID benchmark, and have reported superior performance, i.e., Cumulative Match Characteristic curve (CMC) and Mean Average Precision (MAP), over the state-of-the-art works [Wu et al., 2017], respectively.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/73a532ed6852642a4fb958d5195b486ea6503fad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment (2019, methodology): “The results of cmGAN [3] under different settings on SYSU-MM01 dataset in terms of Rank-1, mAP, and intra-class cosine similarity (ICCS).”
Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-Identification (2020, methodology): “We achieve comparable performance with cmGAN [17] on the SYSU-MM01 dataset with the same ResNet50 baseline, but this method needs more than 2,000 epochs to converge due to the adversarial training process, which is unsuitable for large-scale training.”
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification (2020, methodology): “As a compromise, existing VI-ReID methods mainly focus on learning multi-modal sharable global features, either via one- [7,49,50] or two-stream networks [9,58].”
Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer (2020, methodology): “To alleviate these two problems, we utilize the modality adaptation [4] to filter out modality-specific information from the shared features.”
Channel Augmented Joint Learning for Visible-Infrared Recognition (2021, methodology): “Different from the widely-used bidirectional triplet metric learning [7, 24, 49], we directly optimize the feature embeddings in a mixed batch with the same identity classifier and metric for original visible, infrared and channel augmented modalities.”
Infrared-Visible Cross-Modal Person Re-Identification with an X Modality (2020, methodology): “Compared with other auxiliary structure used methods, like GAN-based ones (Dai et al. 2018; Wang et al. 2019c; 2019a), we implement a much more lightweight and efficient network.”
Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification (2019, methodology): “Recently, adversarial learning [2] or metric learning methods [39, 9, 38] are applied to learn the feature representation of heterogeneous person images involving intra-modality and cross-modality variations.”
Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification (2019, methodology): “As far as we know, all previous methods [21, 22, 23, 2] regard the modality discrepancy δm as a part of the appearance discrepancy δa and attempt to reduce the mixed discrepancy δm+δa using feature-level constraints employed by most conventional re-ID methods.”
Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification (2020, methodology): “(Wu et al. 2017; Ye et al. 2018a; 2018b; Dai et al. 2018; Hao et al. 2019) try to reduce the marginal distribution divergence between RGB and IR modalities, while cannot deal with their joint distributions.”