Joint Detection and Identification Feature Learning for Person Search

By Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, Xiaogang Wang. 2016

898 citations recorded by the source corpus.

Existing person re-identification benchmarks and methods mainly focus on matching cropped pedestrian images between queries and candidates. However, it is different from real-world scenarios where the annotations of pedestrian bounding boxes are unavailable and the target person needs to be searched from a gallery of whole scene images. To close the gap, we propose a new deep learning framework for person search. Instead of breaking it down into two separate tasks—pedestrian detection and person re-identification, we jointly handle both aspects in a single convolutional neural network. An Online Instance Matching (OIM) loss function is proposed to train the network effectively, which is scalable to datasets with numerous identities. To validate our approach, we collect and annotate a large-scale benchmark dataset for person search. It contains 18,184 images, 8,432 identities, and 96,143 pedestrian bounding boxes. Experiments show that our framework outperforms other separate approaches, and the proposed OIM loss function converges much faster and better than the conventional Softmax loss.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “Instead of exhaustively computing these representations every time, we maintain a feature memory bank V for storing them [46].”
Improving Generalization via Scalable Neighborhood Component Analysis (2018, methodology): “There are many metric learning approaches [17,8], some achieving the state-of-the-art performance in image retrieval [47], face recognition [35,40,44], and person re-identification [49].”
Diversity Regularized Spatiotemporal Attention for Video-Based Person Re-identification (2018, methodology): “[44], is built on top of the FC layer to supervise the training of the whole network in an end-to-end fashion.”
Unsupervised Person Re-identification (2017, background): “[31] propose an online instance matching loss to address the problem of having only few training samples each class.”
Proxy Anchor Loss for Deep Metric Learning (2020, background): “Learning a semantic distance metric has been a crucial step for many applications such as content-based image retrieval [14, 21, 27, 29], face verification [18, 25], person re-identification [3, 38], few-shot learning [24, 26, 30], and representation learning [14, 33, 41].”
Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation (2018, methodology): “1, which follows Online Instance Matching (OIM) loss in [35].”
FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, background): “The CalTech [57], MOT17 [22], CUHK-SYSU [58] and PRW [12] datasets provide both box and identity annotations which allows us to train both branches.”
A Bottom-Up Clustering Approach to Unsupervised Person Re-Identification (2019, methodology): “Methods Venue Labels MARS DukeMTMC-VideoReID rank-1 rank-5 rank-10 mAP rank-1 rank-5 rank-10 mAP OIM∗ (Xiao et al. 2017) CVPR18 None 33.”
A Pose-Sensitive Embedding for Person Re-identification with Expanded Cross Neighborhood Re-ranking (2017, neutral): “8 ResNet+OIM [35] CVPR17 - 82.”
Hierarchical Clustering With Hard-Batch Triplet Loss for Person Re-Identification (2019, background): “Zhong et al. propose ECN [37], ECN focuses on exemplar-invariance [28, 29], camera-invariance [36], and neighborhood-invariance [3].”
Track to Detect and Segment: An Online Multi-Object Tracker (2021, background): “This way not only learns an effective embedding as common re-ID loss [45, 25, 32, 51], but also is well compatible with the detection loss and does not hurt detection performance as shown in Tab.”
CrowdHuman: A Benchmark for Detecting Human in a Crowd (2018, background): “Besides, it is a fundamental component for research topics like multiple-object tracking [13], human pose estimation [28], and person search [24].”
Deep Learning for Person Re-Identification: A Survey and Outlook (2020, methodology): “In contrast, some practical open-world applications require end-to-end person search from the raw images or videos [46], [55].”
Mask-Guided Contrastive Attention Model for Person Re-identification (2018, methodology): “try to combine the person detection and identification model [40].”
Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation (2020, background): “, person retrieval [42], human-object interaction [37], and visual reasoning [39].”
Scalable Person Re-identification on Supervised Smoothed Manifold (2017, methodology): “In the furture, we will investigate how to effectively fuse multiple features [38] and apply SSM to other datasets [53].”