QuaterNet: A Quaternion-based Recurrent Model for Human Motion

By Dario Pavllo, David Grangier, Michael Auli. British Machine Vision Conference, 2018

296 citations recorded by the source corpus.

Deep learning for predicting or generating 3D human pose sequences is an active research area. Previous work regresses either joint rotations or joint positions. The former strategy is prone to error accumulation along the kinematic chain, as well as discontinuities when using Euler angle or exponential map parameterizations. The latter requires re-projection onto skeleton constraints to avoid bone stretching and invalid configurations. This work addresses both limitations. Our recurrent network, QuaterNet, represents rotations with quaternions and our loss function performs forward kinematics on a skeleton to penalize absolute position errors instead of angle errors. On short-term predictions, QuaterNet improves the state-of-the-art quantitatively. For long-term generation, our approach is qualitatively judged as realistic as recent neural strategies from the graphics literature.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/643ab168dfa1df777b9276281a54af32bf9b25d2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Human-Computer Interaction

Citations in the corpus

TEMOS: Generating diverse human motions from textual descriptions (2022, background): “While there is a large body of work focusing on future human motion prediction [3, 6, 16, 37, 54, 57] and completion [10, 17], here, we give an overview of methods that generate motions from scratch (i.”
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE (2021, background): “QuaterNet [52] focuses on generating locomotion actions such as walking and running given a ground trajectory and average speed.”
Language2Pose: Natural Language Grounded Pose Forecasting (2019, background): “Short-term predictions [24] focus on modeling joint angles corresponding to hands, legs, head and torso.”
Generating Human Motion from Textual Descriptions with Discrete Representations (2023, methodology): “Approaches mainly focus on efficiently and effectively fusing spatial and temporal information to generate deterministic future motion through different models: RNN [12,20,50,51], GAN [9,30], GCN [49], Attention [48] or even simply MLP [11, 17, 25].”
PhysDiff: Physics-Guided Human Motion Diffusion Model (2022, background): “Early work on motion generation adopts deterministic human motion modeling which only generates a single motion [2, 17, 18, 20, 39, 51, 52, 57, 85, 90].”
Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (2019, background): “…have been considered in various deep learning application areas such as speech recognition (Parcollet et al., 2018b), kinematics/human motion (Pavllo et al., 2018) and computer vision (Gaudet and Maida, 2017), our work is the ﬁrst hypercomplex inductive bias designed for a wide spread of NLP…”
A Spatio-temporal Transformer for 3D Human Motion Prediction (2020, methodology): “Mirroring the joints was previously reported to be useful in [31].”
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction (2020, methodology): “Many methods, including the conventional state-based methods [25, 44, 38, 37, 36] and deep-network-based methods [9, 32, 10, 7, 12, 14, 11, 33, 43], have been proposed to Figure 1.”
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction (2020, background): “Most previous work takes a deterministic approach to modeling human motion and regress a single future motion from past 3D poses [17, 34, 9, 45, 18, 52, 57, 13, 22, 1, 69, 51] or video frames [10, 75, 74].”