Diversity Regularized Spatiotemporal Attention for Video-Based Person Re-identification

By Shuang Li, Slawomir Bak, Peter Carr, Xiaogang Wang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

357 citations recorded by the source corpus.

Video-based person re-identification matches video clips of people across non-overlapping cameras. Most existing methods tackle this problem by encoding each video frame in its entirety and computing an aggregate representation across all frames. In practice, people are often partially occluded, which can corrupt the extracted features. Instead, we propose a new spatiotemporal attention model that automatically discovers a diverse set of distinctive body parts. This allows useful information to be extracted from all frames without succumbing to occlusions and misalignments. The network learns multiple spatial attention models and employs a diversity regularization term to ensure multiple models do not discover the same body part. Features extracted from local image regions are organized by spatial attention model and are combined using temporal attention. As a result, the network learns latent representations of the face, torso and other body parts using the best available image patches from the entire video sequence. Extensive evaluations on three datasets show that our framework outperforms the state-of-the-art approaches by large margins on multiple metrics.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7c25f32e78d0a9ae75825445daf51b064c3f58e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification (2021, neutral): “Another group of methods [32, 48, 26, 8] adopts attention mechanism on video-based person re-identification task to discover key parts in videos.”
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification (2020, methodology): “For person Re-ID, attention is used to combine the spatial-temporal information from different video frames [8,10,20,24].”
Relation-Aware Global Attention for Person Re-Identification (2019, background): “Recently, many studies resort to attention design to address the aforementioned challenges in person re-id by strengthening the discriminative features and suppressing interference [22, 39, 20, 12, 11, 6].”
Discriminability Distillation in Group Representation Learning (2020, background): “It is also beneficial to perform person re-identification [61,39,18,16,17,33,15] from videos.”
Adversarial Text-to-Image Synthesis: A Review (2021, background): “They achieved this by introducing an attention regularization term (inspired by [56, 57]) that only keeps the attention weights for visually important words.”
Question-Guided Hybrid Convolution for Visual Question Answering (2018, background): “The attention mechanisms [17,39] are originally proposed for solving language-related tasks [16].”
Deep Learning for Person Re-Identification: A Survey and Outlook (2020, methodology): “Meanwhile, discriminative [137], diverse [136], consistent [87] and high-order [102] properties are incorporated to enhance the attentive feature learning.”
Robust Anchor Embedding for Unsupervised Video Person re-IDentification in the Wild (2018, background): “Impressive progress has been reported with advanced deep learning methods [22, 34, 51].”
Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis (2019, methodology): “Inspired by the attention regularization employed in recent text embedding networks [21, 34], we propose a new attention regularization term to construct attention weights of words through finding “survival of the fittest”.”
Attention mechanisms in computer vision: A survey (2021, neutral): “[130], [132], [133], [134], [135], [136], [137], [138], [139]”
Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization (2019, methodology): “[29] utilized the diversity loss to deal with occlusions in person re-identification.”