Part-Aligned Bilinear Representations for Person Re-identification

By Yumin Suh, Jingdong Wang, Siyu Tang, Tao Mei, Kyoung Mu Lee. European Conference on Computer Vision, 2018

537 citations recorded by the source corpus.

We propose a novel network that learns a part-aligned representation for person re-identification. It handles the body part misalignment problem, that is, body parts are misaligned across human detections due to pose/viewpoint change and unreliable detection. Our model consists of a two-stream network (one stream for appearance map extraction and the other one for body part map extraction) and a bilinear-pooling layer that generates and spatially pools a part-aligned map. Each local feature of the part-aligned map is obtained by a bilinear mapping of the corresponding local appearance and body part descriptors. Our new representation leads to a robust image matching similarity, which is equivalent to an aggregation of the local similarities of the corresponding body parts combined with the weighted appearance similarity. This part-aligned representation reduces the part misalignment problem significantly. Our approach is also advantageous over other pose-guided representations (e.g., extracting representations over the bounding box of each body part) by learning part descriptors optimal for person re-identification. For training the network, our approach does not require any part annotation on the person re-identification dataset. Instead, we simply initialize the part sub-stream using a pre-trained sub-network of an existing pose estimation network, and train the whole network to minimize the re-identification loss. We validate the effectiveness of our approach by demonstrating its superiority over the state-of-the-art methods on the standard benchmark datasets, including Market-1501, CUHK03, CUHK01 and DukeMTMC, and standard video dataset MARS.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/35f85ab75cb5f982919c0e6f3c8d0e422356ecd5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer (2021, methodology): “Four kinds of methods are compared, which are hand-crafted splitting based Re-ID methods [51, 38], holistic Re-ID methds with key-point information [36, 6], partial ReID methods [57, 10, 12] and occluded ReID methods [13, 5, 30, 11, 40, 60].”
Relation-Aware Global Attention for Person Re-Identification (2019, background): “Some works explore the external clues of human semantics (pose or mask) as attention or to use them to guide the learning of attention [39, 28, 29, 44].”
Pose-Guided Feature Alignment for Occluded Person Re-Identification (2019, methodology): “Recently, some person re-id methods [9, 33, 29, 23, 30] employ pose information to facilitate person re-id models.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, background): “, by capturing the discriminative parts of objects (faces, people, and vehicles) [8], [35], [36], discovering coarse-to-fine”
Hyperbolic Image Embeddings (2019, methodology): “, [46, 59] use verification loss functions based on the Euclidean distance or cosine similarity.”
ABD-Net: Attentive but Diverse Person Re-Identification (2019, unknown): “Many part-based methods have achieved superior performance [2, 3, 26, 27, 28, 4, 5, 29, 6, 30, 31, 8, 32].”
High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification (2020, methodology): “Four kinds of methods are compared, they are vanilla holistic ReID methods [41, 35], holistic ReID methods with key-points information [32, 5], partial ReID methods [45, 8, 9] and occluded ReID methods [12, 49, 10, 26].”
Omni-Scale Feature Learning for Person Re-Identification (2019, background): “In [66, 40, 57, 42, 47, 65], body part-specific CNNs are learned by means of off-the-shelf pose detectors.”
Bilinear Attention Networks for Person Retrieval (2019, methodology): “In person re-ID, the person misalignment [36] and background biases [40] hinder learning a robust representation.”
Batch Coherence-Driven Network for Part-Aware Person Re-Identification (2020, methodology): “training only [15], [27] or during both training and testing [14], [17], [18], [21], [26].”
Memory-Based Neighbourhood Embedding for Visual Recognition (2019, unknown): “3 Part-aligned [33] ECCV’18 69.”
Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer (2020, background): “Many works focus on feature learning [17, 37] and metric learning [3, 24] on the RGB modality.”
Deep Learning for Person Re-Identification: A Survey and Outlook (2020, methodology): “A two-stream network [95] is proposed to extract global appearance and local body part feature maps respectively, and a bilinear-pooling layer is designed to aggregate the two streams to obtain an enhanced representation.”
Joint Discriminative and Generative Learning for Person Re-Identification (2019, methodology): “To further reduce the inﬂuence from intra-class variations, a number of existing methods adopt part-based matching or ensemble to explicitly align and compensate the variations [35,37,47,52,57].”