TransReID: Transformer-based Object Re-Identification

By Shuting He, Haowen Luo, Pichao Wang, Fan Wang, Hao Li, Wei Jiang. 2021

1,247 citations recorded by the source corpus.

Extracting robust feature representation is one of the key challenges in object re-identification (ReID). Although convolution neural network (CNN)-based methods have achieved great success, they only process one local neighborhood at a time and suffer from information loss on details caused by convolution and downsampling operators (e.g. pooling and strided convolution). To overcome these limitations, we propose a pure transformer-based object ReID framework named TransReID. Specifically, we first encode an image as a sequence of patches and build a transformer-based strong baseline with a few critical improvements, which achieves competitive results on several ReID benchmarks with CNN-based methods. To further enhance the robust feature learning in the context of transformers, two novel modules are carefully designed. (i) The jigsaw patch module (JPM) is proposed to rearrange the patch embeddings via shift and patch shuffle operations which generates robust features with improved discrimination ability and more diversified coverage. (ii) The side information embeddings (SIE) is introduced to mitigate feature bias towards camera/view variations by plugging in learnable embeddings to incorporate these non-visual clues. To the best of our knowledge, this is the first work to adopt a pure transformer for ReID research. Experimental results of TransReID are superior promising, which achieve state-of-the-art performance on both person and vehicle ReID benchmarks. Code is available at https://github.com/heshuting555/TransReID.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers (2021, methodology): “Other works have also used Transformers in a variety of tasks such as tracking [61, 62], super-resolution [63], ReID [64], Colorization [65], Retrieval [66] and multi-modal learning [67, 68].”
Multiscale Vision Transformers (2021, background): “An emerging thread of work aims at applying transformers to vision tasks such as object detection [5], semantic segmentation [121, 105], 3D reconstruction [77], pose estimation [113], generative modeling [17], image retrieval [30], medical image segmentation [16,103,117], point clouds [45], video in”
A Survey on Vision Transformer (2020, background): “[90] proposed TransReID to investigate the application of pure transformers in the field of object re-identification (ReID).”
CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels (2022, methodology): “Our method achieves 63.0% mAP and 84.4% R1 on the CNNbased backbone, and 73.4% mAP and 88.7% R1 (6.0% and 3.4% higher than Transreid+SIE+OLP) on the ViT-based backbone using only the CLIP-ReID method, in further use of SIE and OLP we can improve mAP and R1 to 75.8% and 89.7%.”
Patch-based Knowledge Distillation for Lifelong Person Re-Identification (2022, methodology): “Person ReID has made remarkable progress under various settings such as supervised learning [18, 40, 57], unsupervised learning [9, 36] and unsupervised domain adaptation [12, 15, 66, 75, 76].”
PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (2021, unknown): “Many other ViT variants [8, 13, 21, 22, 25, 37, 53, 59, 69] are proposed from then, which achieve promising performance compared with its counterpart CNNs for image analysis tasks [6, 23, 73].”
Residual Mixture of Experts (2022, background): “Triggered by ViT, a series of follow-up works [8, 9, 14, 17, 21, 25, 30, 38, 49, 50, 52, 54, 57, 59, 60] continuously refresh the image classification records and push the potential of vision transformers in computer vision to a new height.”
Multi-animal pose estimation, identification and tracking with DeepLabCut (2022, methodology): “To track unidentified animals we turn to metric learning 4 with transformers, which are state-of-the-art for reID of humans and vehicles 51 .”
One Venue, Two Conferences: The Separation of Chinese and American Citation Networks (2022, unknown): “However, similar datasets like Market-1501 [18] from Tsinghua University are still being actively used [5, 4].”
Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks (2021, background): “object re-identification [41], image captioning [42], point cloud learning [43] and self-supervised learning [44].”
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition (2023, unknown): “semantic segmentation [13], [28], [42], [48], [66], [112], [113], and re-identification [12], [36], [108].”
UniFormer: Unifying Convolution and Self-Attention for Visual Recognition (2022, background): “Besides, many efforts have been devoted to applying ViTs for various vision tasks, including object detection [8], [111], semantic segmentation [14], [43], [95], pose estimation [51], [98], [102], re-identification [36], and low-level image processing [12], [53].”
MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation (2021, background): “Recently, Transformer [37] equipped with the powerfully global self-attention mechanism has received increasingly research interest in the computer vision community [10,19,26,44].”
TransFG: A Transformer Architecture for Fine-grained Recognition (2021, methodology): “(He et al. 2021) proposed TransReID which embedded side information into transformer along with the JPM to boost the performance on object re-identification.”