A Deep Learning Framework for Character Motion Synthesis and Editing

By Daniel Holden, Jun Saito, Taku Komura. ACM Transactions on Graphics, 2016

738 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/f67d67ea1b416c5dd5f103f684abce9c20ef9ddb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory (2022, background): “In recent years, with the emergence of deep learning, many works design a dedicated network structure, including CNNs [14], RNNs [37, 3, 40, 15], GCNs [41, 35, 9], GANs [25, 36] and Transformers [29, 30, 28], to map the given music to a joint sequence of the continuous human pose space directly.”
EDGE: Editable Dance Generation From Music (2022, background): “Gener-ating human motion conditioned on various inputs—e.g., joystick control [34], class-conditioning [15, 43], text-to-motion [44,63], seed motions [8,22,46,62]—is also an active area of study.”
Hierarchical Style-based Networks for Motion Synthesis (2020, methodology): “Comparison of motion naturalness with DeepSyn [11] and QuaterNet [25].”
Phase-functioned neural networks for character control (2017, methodology): “Previously some success has been achieved using convolutional models such as CNNs [Holden et al. 2016, 2015], autoregressive models such as RBMs [Taylor and Hinton 2009], and RNNs [Fragkiadaki et al. 2015].”
QuaterNet: A Quaternion-based Recurrent Model for Human Motion (2018, methodology): “Compared to other work with positional loss [23, 24], our strategy penalizes position errors properly and avoids re-projection onto skeleton constraints.”
Deep Representation Learning for Human Motion Prediction and Classification (2017, methodology): “While the idea of manifold learning resembles our approach, the use of convolutional and pooling layers prevents the implementation of deeper hierarchies due to blurring effects [11].”
Language2Pose: Natural Language Grounded Pose Forecasting (2019, neutral): “[14].”
Mode-adaptive neural networks for quadruped motion control (2018, background): “They also demonstrate CNN’s capabilities to map high-level user instructions to character movements [Holden et al. 2016].”
Deep Audio-visual Learning: A Survey (2020, methodology): “The motion synthesis methods leveraged multiple techniques, such as dimensionality reduction [103, 104] , hidden Markov models [105] , Gaussian processes [106] , and neural networks [107, 108, 109] .”
TEMOS: Generating diverse human motions from textual descriptions (2022, methodology): “[20], which is used in the previous work we compare with [2, 13].”
Character controllers using motion VAEs (2020, methodology): “Time-convolutional autoencoders [Holden et al. 2016] have been used to first learn a latent motion manifold, from task-relevant data.”
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ (2021, background): “With the development in deep learning, many approaches explore the applicability of neural networks to generate 3D motion by training on a large-scale motion capture dataset, where network architectures such as CNNs [35, 34], GANs [31], RBMs [80], RNNs [24, 4, 40, 27, 16, 18, 88, 12, 87] and Transfo”
A Spatio-temporal Transformer for 3D Human Motion Prediction (2020, methodology): “In other words, the outputs may contain artifacts such as inconsistent bone lengths across the frames [14, 20].”
Generating Human Motion from Textual Descriptions with Discrete Representations (2023, methodology): “Holden et al. [34] learn a convolutional autoencoder to reconstruct motion, the learned latent representation can be used to synthesize and edit motion.”
GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping (2021, methodology): “Most methods generate the motion of 3D skeletons [13, 22, 38–40, 61 ], while others [ Motion generation for bodies in 3D scenes: Most early methods extend MoCap databases with point annotations for foot and hand contact [12, 25, 30, 31].”
MotionCLIP: Exposing Human Motion Generation to CLIP Space (2022, methodology): “Holden et al. [2016] suggested to code style using the latent code’s Gram matrix, inspired by Gatys et al. [2016].”
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE (2021, background): “Several works [27, 33, 63] learn a motion manifold, and use it for motion denoising, which is one of our use cases.”
Robust motion in-betweening (2020, background): “Deep neural networks can circumvent these limitations by allowing huge, heterogeneous datasets to be used for training, while having a fixed computation budget at run-time.”