History Repeats Itself: Human Motion Prediction via Motion Attention
By Wei Mao, Miaomiao Liu, Mathieu Salzmann. European Conference on Computer Vision, 2020
361 citations recorded by the source corpus.
Human motion prediction aims to forecast future human poses given a past motion. Whether based on recurrent or feed-forward neural networks, existing methods fail to model the observation that human motion tends to repeat itself, even for complex sports actions and cooking activities. Here, we introduce an attention-based feed-forward network that explicitly leverages this observation. In particular, instead of modeling frame-wise attention via pose similarity, we propose to extract motion attention to capture the similarity between the current motion context and the historical motion sub-sequences. Aggregating the relevant past motions and processing the result with a graph convolutional network allows us to effectively exploit motion patterns from the long-term history to predict the future poses. Our experiments on Human3.6M, AMASS and 3DPW evidence the benefits of our approach for both periodical and non-periodical actions. Thanks to our attention model, it yields state-of-the-art results on all three datasets. Our code is available at this https URL.
Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering
View Paper: https://www.semanticscholar.org/paper/4a00f94559efeca391d4e9352a0a72baacb2bc7e
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Michael J. Black
- Dimitrios Tzionas
- Vasileios Choutas
- Otmar Hilliges
- Omid Taheri
- Luc Van Gool
- Jitendra Malik
- Xiaodong Cun
In Control and Systems Engineering
- Michael J. Black
- Dimitrios Tzionas
- Omid Taheri
- Vasileios Choutas
- Otmar Hilliges
- Manuel Kaufmann
- Peng Cao
- Emre Aksan
Citations in the corpus
- A Spatio-temporal Transformer for 3D Human Motion Prediction (2020, methodology): “Recent works [5, 24, 36] model the temporal aspects of 3D human motion by encoding every joint’s trajectory with the discrete cosine transformation (DCT).”
- Generating Human Motion from Textual Descriptions with Discrete Representations (2023, methodology): “Approaches mainly focus on efficiently and effectively fusing spatial and temporal information to generate deterministic future motion through different models: RNN [12,20,50,51], GAN [9,30], GCN [49], Attention [48] or even simply MLP [11, 17, 25].”
- GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping (2021, methodology): “However, even recent methods [38, 46, 61, 64], mostly study the body “in isolation”; i.e., with no scene context.”