Style translation for human motion

By Eugene Hsu, Kari A. Pulli, Jovan Popovic. ACM Transactions on Graphics, 2005

393 citations recorded by the source corpus.

Style translation is the process of transforming an input motion into a new style while preserving its original content. This problem is motivated by the needs of interactive applications, which require rapid processing of captured performances. Our solution learns to translate by analyzing differences between performances of the same content in input and output styles. It relies on a novel correspondence algorithm to align motions, and a linear time-invariant model to represent stylistic differences. Once the model is estimated with system identification, our system is capable of translating streaming input with simple linear operations at each frame.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/1cd6d6cb70ed8a3bb2c13fc6aabf907d620b527f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Sharing Features among Dynamical Systems with Beta Processes (2009, methodology): “The linear dynamical system is a common model for describing simple human motion [ 11 ], and the more complicated SLDS has been successfully applied to the problem of human motion synthesis, classification, and visual tracking [17, 18].”
Motion graphs++ (2012, background): “In general, previous generative statistical models can be categorized into variants of Hidden Markov Models (HMMs) [Molina Tanco and Hilton 2000; Bowden 2000; Brand and Hertzmann 2000], statistical dynamic models for modeling spatial-temporal variations within a small temporal window [Li et al. 2002”
Recurrent Network Models for Human Dynamics (2015, methodology): “Prior work has tackled synthesis of stylized human motion using bilinear spatiotemporal basis models [1], Hidden Markov Models [3], linear dynamical systems [21], and Gaussian process latent variable models [46, 40], as well as multilinear variants thereof [12, 45].”
Hierarchical Convolutional Deep Learning in Computer Vision (2013, background): “This includes motionstyle transfer [53], economic forecasting with external indicators [86], and various tasks in natural language processing [24].”
Learning GP-BayesFilters via Gaussian process latent variable models (2009, methodology): “It has been applied successfully for human motion animation (Hsu et al. 2005).”
Two Distributed-State Models For Generating High-Dimensional Time Series (2011, methodology): “The second data set usedin th se experiments was released by Hsu et al. (2005).”
Generalizing motion edits with Gaussian processes (2009, methodology): “One mitigating strategy in motion transfer is to use a heuristic to detect whether an input frame is outside the space of training data, then blend the output frame with the input [Hsu et al. 2005].”
Factored conditional restricted Boltzmann Machines for modeling motion style (2009, methodology): “We use the dataset .rst proposed by (Hsu et al., 2005) which consists of labeled sequences of seven types of walking: (crouch, jog, limp, normal, side-right, sway, waddle) each at three different speeds (slow, medium, fast).”
Learning Stochastic Recurrent Networks (2014, background): “The motion capture data set [25, 21] is a sequence of kinematic quantities obtained from a human body during walking.”
Recycle-GAN: Unsupervised Video Retargeting (2018, background): “A variety of work dealing with image-to-image translation [11,17,23,40,53] and style translation [4,10,19] exists.”
Exploiting compositionality to explore a large space of model structures (2012, background): “We ﬁrst consider a human motion capture dataset (Hsu et al., 2005; Taylor et al., 2007) consisting of a person walking in a variety of styles.”
Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription (2012, background): “The human motion capture dataset 2 is represented by a sequence of joint angles, translations and rotations of the base of the spine in an exponential-map parameterization (Hsu et al., 2005; Taylor et al., 2007).”
Modeling Human Motion Using Binary Latent Variables (2006, methodology): “We used data from the CMU Graphics Lab Motion Capture Database as well as from [12] (see acknowledgments).”
Mode-adaptive neural networks for quadruped motion control (2018, methodology): “Similar to motion interpolation, methods based on linear regression [Hsu et al. 2005; Xia et al. 2015], KNN [Lee et al. 2010] and kernel-based approaches [Wang et al. 2008] are proposed.”
Documentation Mocap Database HDM05 (2007, methodology): “…on shelf (top) [6] turn around left and walk 3 steps away from shelf [7] turn around left and walk 3 steps to shelf [8] grab item from shelf (top) [9] turn around left and walk 3 steps away from shelf [10] turn around left and walk 3 steps to shelf [11] deposit item on shelf (bottom) [12] turn…”
Neural Granger Causality (2021, methodology): “While the focus of previous work has been on motion classiﬁcation [62] and segmentation [37], our analysis delves into the potentially long-range, non-linear dependencies between different regions of the body during natural motion behavior.”