Spindle Net: Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion

By Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, Xiaoou Tang. Computer Vision and Pattern Recognition, 2017

870 citations recorded by the source corpus.

Person re-identification (ReID) is an important task in video surveillance and has various applications. It is non-trivial due to complex background clutters, varying illumination conditions, and uncontrollable camera settings. Moreover, the person body misalignment caused by detectors or pose variations is sometimes too severe for feature matching across images. In this study, we propose a novel Convolutional Neural Network (CNN), called Spindle Net, based on human body region guided multi-stage feature decomposition and tree-structured competitive feature fusion. It is the first time human body structure information is considered in a CNN framework to facilitate feature learning. The proposed Spindle Net brings unique advantages: 1) it separately captures semantic features from different body regions thus the macro-and micro-body features can be well aligned across images, 2) the learned region features from different semantic regions are merged with a competitive scheme and discriminative features can be well preserved. State of the art performance can be achieved on multiple datasets by large margins. We further demonstrate the robustness and effectiveness of the proposed Spindle Net on our proposed dataset SenseReID without fine-tuning.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ee62cff7f027a31c3bd7f313bfdc04e7d79eb04a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Yi Yang
2. Liang Zheng
3. Qi Tian
4. Zhedong Zheng
5. Shiliang Zhang
6. Tao A. Xiang
7. Liang Wang
8. G. Wang

In Computer Vision and Pattern Recognition

1. Yi Yang
2. Luc Van Gool
3. Xiaogang Wang
4. Dacheng Tao
5. Ming-Hsuan Yang
6. Tao A. Xiang
7. Qi Tian
8. Ling Shao

Citations in the corpus

• Pose Transferrable Person Re-identification (2018, methodology): “Zhao et al [42] propose a novel DCNN named as Spindle Net to fuse whole body feature and body region feature, and Li et al [17] design a Multi-Scale Context-Aware Network to extract small visual cues that may be very useful to distinguish the pedestrian pairs.”
• Pose-Normalized Image Generation for Person Re-identification (2017, methodology): “For example, [39, 57] first detect normalized part regions from a person image, and then fuse the features extracted from the original images and the part region images.”
• Patch-based Knowledge Distillation for Lifelong Person Re-Identification (2022, background): “It consists of twelve ReID datasets, including five seen datasets(Market-1501[73],CUHK-SYSU[64],DukeMTMC-reID[48], MSMT17_V2[61]andCUHK03[32])fornon-forgettingevaluation andsevenunseendatasets(VIPeR[16],PRID[22],GRID[39],i-LIDS[74],CUHK01[31],CUHK02[30]andSenseReID[70])for generalizationevaluation.”
• Part-Aligned Bilinear Representations for Person Re-identification (2018, methodology): “[16] used 13 inception modules, whereas we use 7.”
• Person Transfer GAN to Bridge Domain Gap for Person Re-identification (2017, background): “In [37], more fine-grained part regions are localized and then fed into the proposed Spindle Net for descriptor learning.”
• Cross-Domain Adversarial Feature Learning for Sketch Re-identification (2018, methodology): “For the feature learning network, some researchers have adopted a classical convolution neural network (CNN) like GoogleNet [25] and ResNet [11] while others have designed their network architectures for particular purposes [34].”
• Relation-Aware Global Attention for Person Re-Identification (2019, background): “Some works explore the external clues of human semantics (pose or mask) as attention or to use them to guide the learning of attention [39, 28, 29, 44].”
• FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification (2018, methodology): “[2] proposed SpindleNet for learning person features of different body regions with additional human pose information.”
• Mask-Guided Contrastive Attention Model for Person Re-identification (2018, methodology): “ated by either part region detection [24], or pose and keypoints estimation [32, 23, 49, 44].”
• Distilled Person Re-Identification: Towards a More Scalable System (2019, background): “Person Reidentification has witnessed a fast growing development in recent years, from feature design [19, 15, 32, 31, 35, 69, 58] to distance metric learning [54, 19, 43, 25, 67, 36, 41, 30, 58, 33, 38, 31, 9, 68, 60, 63, 69, 29, 52, 6] and end-to-end deep learning [28, 1, 55, 57, 49, 22, 64, 65, 7”
• In Defense of the Triplet Loss for Person Re-Identification (2017, methodology): “Furthermore, we do not rely on network components specifically tailored towards person ReID, but train a plain feed-forward CNN, unlike many other approaches that train from scratch [21, 1, 39, 42, 34, 20, 48].”
• Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-identification (2017, neutral): “Recently, CNN-based features [2,21,22,27] have also achieved great progress on person ReID.”
• Human Semantic Parsing for Person Re-identification (2018, methodology): “[45] use region proposal network, trained on an auxiliary pose dataset, to detect body parts.”
• Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition (2021, methodology): “tion ability of the EfficientGCN, we apply the EfficientGCNB4 to other skeleton-based tasks such as person re-identification (ReID) [54], [55].”