Recurrent Attention Models for Depth-Based Person Identification

By Albert Haque, Alexandre Alahi, Li Fei-Fei. 2016

152 citations recorded by the source corpus.

We present an attention-based model that reasons on human body shape and motion dynamics to identify individuals in the absence of RGB information, hence in the dark. Our approach leverages unique 4D spatio-temporal signatures to address the identification problem across days. Formulated as a reinforcement learning task, our model is based on a combination of convolutional and recurrent neural networks with the goal of identifying small, discriminative regions indicative of human identity. We demonstrate that our model produces state-of-the-art results on several published datasets given only depth images. We further study the robustness of our model towards viewpoint, appearance, and volumetric changes. Finally, we share insights gleaned from interpretable 2D, 3D, and 4D visualizations of our model's spatio-temporal attention.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Unsupervised Learning of Long-Term Motion Dynamics for Videos (2017, methodology): “[4] proposed an alternative to skeleton representation by using a Recurrent Attention model (RAM).”
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals (2017, methodology): “[24] use a Recurrent Attention Model (RAM) to re-identify humans.”
Reinforced Temporal Attention and Split-Rate Transfer for Depth-Based Person Re-identification (2017, methodology): “The small sample size problem is especially acute in person re-identification from temporal sequences [26,86,9,54,110], as the feature dimensionality increases linearly in the number of frames that are accumulated compared to the singleshot representations.”
Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community (2017, unknown): “Haque et al.(128) developed an attention-based human”
Towards Viewpoint Invariant 3D Human Pose Estimation (2016, background): “Other features such as viewpoint invariant 3D feature maps [43], histograms of 3D joint locations [63], multifractal spectrum [64], volumetric attention models [28], and volumetric convolutional filters [45,46] have been proposed for 3D modeling.”
Multi-attention Network for One Shot Learning (2017, background): “Attention models have been applied to a variety of computer vision problems including image classification [25, 1, 8], semantic segmentation [3], visual tracking [5], person identification [9], image captioning [4, 6, 26] and question answering [27].”
Deep Learning for Person Re-Identification: A Survey and Outlook (2020, background): “A recurrent attention-based model is proposed in [183] to address the depth-based person identiﬁcation.”
Attentive Systems: A Survey (2017, background): “Task-driven saliency has shown to benefit many applications such as image question and answering (Yang et al. 2016), and person identification (Haque et al. 2016), handwriting synthesis (Graves 2013), machine translation (Bahdanau et al. 2014), image caption generation (Xu et al. 2015) and visual…”
Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition (2018, methodology): “[33] applied DRL to person identification, Yeung et al.”